问题描述

今天发现当天的索引在ES中并没有创建，logstash中不停的报错：

[2021-05-24T05:47:51,904][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-05-24T05:47:51,904][INFO ][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>1}

索引变成了只读：

原因

一旦在存储超过95％的磁盘中的节点上分配了一个或多个分片的任何索引，该索引将被强制进入只读模式

解决办法

经过分析，发现是因为ES所在服务器磁盘空间太低引起，具体解决办法如下：

第一种办法：在kibana开发控制台执行下面语句：

PUT _settings
{
  "index": {
    "blocks": {
      "read_only_allow_delete": null
    }
  }
}

第二种办法：在es节点服务器上执行：

curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'