Introduce query timeout configuration (#2157)

## Summary

Implements configurable query execution timeout controls to prevent
poorly optimized or excessive queries from consuming excessive server
resources, causing performance degradation, or crashing the Zabbix
server.

Fixes: https://github.com/grafana/oss-big-tent-squad/issues/127

## Problem

Previously, the plugin only had an HTTP connection timeout (`timeout`)
that controlled individual API request timeouts. However, a complete
query execution could involve multiple API calls and run indefinitely if
not properly controlled, potentially causing resource exhaustion.

## Solution

Added a new `queryTimeout` setting that enforces a maximum execution
time for entire database queries initiated by the plugin. Queries
exceeding this limit are automatically terminated with proper error
handling and logging.

## Testing

1. Configure a datasource with `queryTimeout` set to a low value (e.g.,
5 seconds)
2. Execute a query that would normally take longer than the timeout
3. Verify that:
   - Query is terminated after the timeout period
   - Error message indicates timeout occurred
   - Logs contain timeout warning with query details
   - Other queries in the same request continue to execute

## Notes

- `queryTimeout` is separate from `timeout` (HTTP connection timeout)
- `queryTimeout` applies to the entire query execution, which may
involve multiple API calls
- Default value of 60 seconds ensures reasonable protection while
allowing normal queries to complete
- Timeout errors are logged with query refId, queryType, timeout
duration, and datasourceId for troubleshooting
This commit is contained in:
ismail simsek
2026-01-12 15:30:31 +01:00
committed by GitHub
parent 7eb80d3f23
commit a2f8b6433a
7 changed files with 366 additions and 50 deletions

View File

@@ -9,15 +9,21 @@ const (
// ZabbixDatasourceSettingsDTO model
type ZabbixDatasourceSettingsDTO struct {
AuthType string `json:"authType"`
Trends bool `json:"trends"`
TrendsFrom string `json:"trendsFrom"`
TrendsRange string `json:"trendsRange"`
CacheTTL string `json:"cacheTTL"`
Timeout interface{} `json:"timeout"`
AuthType string `json:"authType"`
Trends bool `json:"trends"`
TrendsFrom string `json:"trendsFrom"`
TrendsRange string `json:"trendsRange"`
CacheTTL string `json:"cacheTTL"`
// Timeout is the HTTP client connection timeout in seconds for individual API requests to Zabbix.
// This controls how long to wait for a single HTTP request/response cycle. Default is 30 seconds.
Timeout interface{} `json:"timeout"`
// QueryTimeout is the maximum execution time in seconds for entire database queries initiated by the plugin.
// This controls the total time allowed for a complete query execution (which may involve multiple API calls).
// Queries exceeding this limit will be automatically terminated. Default is 60 seconds.
QueryTimeout interface{} `json:"queryTimeout"`
DisableDataAlignment bool `json:"disableDataAlignment"`
DisableReadOnlyUsersAck bool `json:"disableReadOnlyUsersAck"`
DisableDataAlignment bool `json:"disableDataAlignment"`
DisableReadOnlyUsersAck bool `json:"disableReadOnlyUsersAck"`
}
// ZabbixDatasourceSettings model
@@ -27,8 +33,14 @@ type ZabbixDatasourceSettings struct {
TrendsFrom time.Duration
TrendsRange time.Duration
CacheTTL time.Duration
Timeout time.Duration
// Timeout is the HTTP client connection timeout for individual API requests to Zabbix.
// This controls how long to wait for a single HTTP request/response cycle. Default is 30 seconds.
Timeout time.Duration
// QueryTimeout is the maximum execution time for entire database queries initiated by the plugin.
// This controls the total time allowed for a complete query execution (which may involve multiple API calls).
// Queries exceeding this limit will be automatically terminated. Default is 60 seconds.
QueryTimeout time.Duration
DisableDataAlignment bool `json:"disableDataAlignment"`
DisableReadOnlyUsersAck bool `json:"disableReadOnlyUsersAck"`
DisableDataAlignment bool `json:"disableDataAlignment"`
DisableReadOnlyUsersAck bool `json:"disableReadOnlyUsersAck"`
}