Asyncopenai Concurrency Httpx Pool

kennethkhoocy/asyncopenai-concurrency-httpx-pool

| Raise real concurrency in asyncio LLM batch scorers built on the OpenAI SDK (1) raising an asyncio.Semaphore above ~100 produces no throughput gain, (2) a batch pipeline saturates near 100 in-flight requests despite a larger semaphore, (3) planning a high-concurrency campaign against a provider with AsyncOpenAI's default httpx pool caps max_connections at 100, silently bottlenecking any larger semaphore — you must pass a custom http_client with httpx.Limits sized to the semaphore.