[Bug]: ElasticSearchQuery does not resetting
Pre-submission checklist
- I have searched the existing issues and this bug has not already been reported.
UnoPim Version(s) affected
1.0.0 - 2.1.6
Issue Description
When run long import with 10k+ products, after some time appears error: "Import failed {"exception":"[object] (Elastic\Elasticsearch\Exception\ClientResponseException(code: 400): 400 Bad Request: {"error":{"root_cause":[{"type":"query_s hard_exception","reason":"failed to create query: maxClauseCount is set to 40329","index_uuid":"i9qAyV1TRO6MIRjJi_tkOg","index":"uno-_products"}],"type":"search_phase_execution_exception","reason ":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index" ...." After investigation I found reason: There are UniqueAttributeValue class validate() method `public function validate(string $attribute, mixed $value, \Closure $fail): void { $queryBuilder = app(ElasticProductQueryBuilder::class);
$value = ! is_array($value) ? [$value] : $value;
$queryBuilder->applyFilter($this->attributeCode, FilterOperators::IN, $value);
if ($this->productId) {
$queryBuilder->applyFilter('product_id', FilterOperators::NOT_EQUAL, $this->productId);
}
$esQuery = ElasticSearchQuery::build();
$results = ResultCursorFactory::createCursor($esQuery, ['pagination' => ['per_page' => 1]]);
if (count($results->getAllIds()) > 0) {
$fail(trans('validation.unique'));
}
}`
- validate calls for every product in import, validating unique fields with Elasticsearch
- here's queey for first product in list
{ "index": "uno-_products", "body": { "query": { "constant_score": { "filter": { "bool": { "filter": [ { "terms": { "values.common.url_key-text": [ "793798858-columbus-avtomobilna-markiza-tent-columbus-25x20-m" ] } } } ], "must_not": [ { "term": { "id": 71063 } } } ] } } } }, "track_total_hits": true, "size": 1, "from": 0, "sort": { "id": "asc" }, "stored_fields": [] } }validate() for second product send next query:{ "index": "uno-_products", "body": { "query": { "constant_score": { "filter": { "bool": { "filter": [ { "terms": { "values.common.url_key-text": [ "793798859-columbus-avtomobilna-markiza-tent-columbus-25x20-m" ] } }, { "terms": { "values.channel_locale_specific.default.uk_UA.slug-text": [ "793798858-columbus-avtomobilna-markiza-tent-columbus-25x20-m-uk_UA" ] } } ], "must_not": [ { "term": { "id": 71063 } }, { "term": { "id": 71065 } } ] } } } }, "track_total_hits": true, "size": 1, "from": 0, "sort": { "id": "asc" }, "stored_fields": [] } }after 1000-th product call, query should contain 1000 previous conditions, and later fail with error. As temporary solution I'd added reset() method to ElasticSearchQuerypublic function reset(): self { $this->excludeConditions = []; $this->filterConditions = []; $this->orConditions = []; $this->includeConditions = []; $this->sortingConditions = []; $this->aggregationConditions = []; return $this; }and changed validation() topublic function validate(string $attribute, mixed $value, \Closure $fail): void { $queryBuilder = app(ElasticProductQueryBuilder::class); ElasticSearchQuery::reset();
Preconditions (environment)
- PHP version: 8.3
- Database (MySQL/PostgreSQL + version): MariaDB
- Elasticsearch version (or "not used"): 9.1.2
- Web server: Octane
- OS: Ubuntu
- Browser: Chrome
Steps to reproduce
- Run long import
- Import fail on high product numbers
Expected Result
Import do not fail
Actual Result
Import fail
Screenshots
No response
Additional Context
No response
Source: unopim/unopim