OpenSearch can enforce read-only states on clusters or indices to protect against issues like low disk space or cluster instability. Understanding and resolving these blocks is crucial for maintaining a healthy and operational cluster. Below is a guide to address common scenarios.
How to Resolve cluster.blocks.read_only
The
cluster.blocks.read_only setting typically occurs when OpenSearch detects a critical issue, such as running out of disk space. The cluster automatically sets itself to read-only to prevent further damage.Steps to resolve:
- Free up disk space:
-
- Delete unnecessary indices.
- Archive old data.
- Increase disk capacity.
- Clear the read-only block: Run the following command to remove the block after resolving the disk space issue:
PUT _cluster/settings
{
"persistent": {
"cluster.blocks.read_only": false
}
}Index-Level Read-Only Block
At the index level, OpenSearch can set specific indices to read-only, preventing writes. This is often triggered by disk watermarks or manual configuration.
Steps to resolve:
- Identify the affected indices using:
GET _cat/indices?v 2. Clear the block for an individual index:
PUT /<index-name>/_settings
{
"index.blocks.read_only": false
} 3. If multiple indices are affected, use a wildcard:
PUT /*/_settings
{
"index.blocks.read_only": false
}How to Resolve cluster.blocks.read_only_allow_delete
This block allows only delete operations on indices, typically to help free up space when disk watermarks are exceeded.
Steps to resolve:
Steps to resolve:
- Free up disk space as described earlier.
- Clear the
read_only_allow_deleteblock with:
PUT _cluster/settings
{
"persistent": {
"cluster.blocks.read_only_allow_delete": false
}
} Index-Level Read-Only Delete Block
When indices are individually set to
Steps to resolve:
read_only_allow_delete, write operations are blocked except for deletions.Steps to resolve:
- Check index settings:
GET /<index-name>/_settings 2. Remove the block for a specific index:
PUT /<index-name>/_settings
{
"index.blocks.read_only_allow_delete": false
} 3. To remove the block from all indices:
PUT _all/_settings
{
"index.blocks.read_only_allow_delete": false
}Conclusion
Understanding and resolving read-only blocks in OpenSearch is crucial for restoring normal operations. Always monitor disk usage and cluster health to prevent these issues from recurring.
The post OpenSearch: Cluster Blocks Read-Only appeared first on SOC Prime.
Related posts:
Mistakes to Mastery: Get to Know Phishing Defense Coaching from Huntress SAT | Huntress
SlashAndGrab: ScreenConnect Post-Exploitation in the Wild (CVE-2024-1709 & CVE-2024-1708) | Huntress
Tagging or Classifying Your OSINT Collection
(In)tuned to Takeovers: Abusing Intune Permissions for Lateral Movement and Privilege Escalation in Entra ID Native Environments
Leave a Reply