ISC: Can't query the number of failed workflow executions in an workflow

What problem are you observing?

You can NOT design a workflow which iterates all existing workflows and counts how many executions have failed.

As reported by me in April in Workflows: query failed workflow executions (in another workflow) , and @german_duci in October, you often get a HTTP 502 error when using the count parameter.

What is the correct behavior?

The API should not throw a HTTP 502 Bad Gateway error.

What product feature is this related to?

ISC Workflow Executions API in combination with ISC Workflows.

What are the steps to reproduce the issue?

  • Create ~20 workflows, execute them ~200 each.
  • Create a workflow like this, where you list all workflows, and for each workflow you count the number of failed executions.

  • Execute the workflow and check how many API calls returned a status code of 502.
         "hTTPRequest1": {
          "body": "",
          "headers": {},
          "responseTime": "40.943339 seconds",
          "statusCode": 502
        },

Issues that can be reproduced are more likely to be prioritized.

But how likely?

Do you have any other information about your environment that may help?

No.


The issue:

  • Executing /workflows/:id/executions?filters=status eq "Failed" takes about 50-90ms.
  • Executing /workflows/:id/executions?filters=status eq "Failed"&count=true takes a whooping 1.2 - 1.6 seconds.
  • Executing the APIs in a workflow loop seem to execute them in parallel, which seems to block some internal structure and the responses take 20-50 seconds, and they return with error code 502.