I’m trying to create a connector which needs to pull all the users in my organization. The aggregation is only able to retrieve 10k records because of the limitation in the endpoint. So, one alternative I have is to subscribe to kafka topics. I want to know if there is any known way of integrating and subscribing to kafka topics through workflows or connectors.
Is the limitation was set by Kafka vendor? then do they have any other alternative to pull more than 10k accounts via API itself (not via kafka topics)
The limitation was set to maintain performance. So other option we have is to listen to kafka topics. I’m trying to research if that is even feasible option.