Duplicate Accounts search query returning too much data

[Abhinov7 (Abhinov Dhonthula)]

Hi All,

We are working on finding the duplicate accounts and automate it using workflows.

Using below body we are able to extract the duplicate accounts in tenant.

{
	"query": {
		"query": "*"
	},
	"indices": [
		"identities"
	],
	"aggregationsDsl": {
		"accounts": {
			"nested": {
				"path": "accounts"
			},
			"aggs": {
				"source_name": {
					"terms": {
						"field": "accounts.source.name.exact",
						"min_doc_count": 2,
						"size": 1000
					},
					"aggs": {
						"identities": {
							"terms": {
								"field": "_id",
								"min_doc_count": 2,
								"size": 1000
							},
							"aggs": {
								"accounts": {
									"top_hits": {
                                        
                                    }
								}
							}
						}
					}
				}
			}
		}
	}
}

This is producing too much information and workflow is throwing timeout error. Is there a way to exclude hits data in the output.

Thanks,
Abhinov

[colin_mckibben]

You can add the queryResultFilter property to limit which fields are returned, greatly reducing the amount of data returned. Simply add the following section to your payload and it will minimize the hits data size.

"queryResultFilter": {
        "includes": [
            "aggregations"
        ]
    },

The full payload will look like this:

{
	"query": {
		"query": "*"
	},
	"indices": [
		"identities"
	],
    "queryResultFilter": {
        "includes": [
            "aggregations"
        ]
    },
	"aggregationsDsl": {
		"accounts": {
			"nested": {
				"path": "accounts"
			},
			"aggs": {
				"source_name": {
					"terms": {
						"field": "accounts.source.name.exact",
						"min_doc_count": 2,
						"size": 1000
					},
					"aggs": {
						"identities": {
							"terms": {
								"field": "_id",
								"min_doc_count": 2,
								"size": 1000
							},
							"aggs": {
								"accounts": {
									"top_hits": {
                                        
                                    }
								}
							}
						}
					}
				}
			}
		}
	}
}

[Abhinov7 (Abhinov Dhonthula)]

Hi @colin_mckibben ,

Thank you so much for the response.

Its working as expected.

Thanks,
Abhinov