libb.RateLimitedExecutor

class RateLimitedExecutor(max_workers=None, max_per_second=inf, show_progress=False)

Bases: object

Thread pool executor with rate limiting and Request/Response API.

Provides clean request/response API where every response includes the original request for easy result tracking and exception handling.

Basic Usage:

with RateLimitedExecutor(max_workers=10, max_per_second=5, show_progress=True) as executor:
    responses = executor.execute_items(process_fn, items, desc='Processing')
    for response in responses:
        if response.success:
            print(f"Item {response.request.id}: {response.result}")
        else:
            print(f"Item {response.request.id} failed: {response.exception}")

Advanced Usage with Custom IDs:

requests = [TaskRequest(item=x, id=f'custom_{i}') for i, x in enumerate(items)]
responses = executor.execute(process_fn, requests, desc='Processing')
result_map = {r.request.id: r.result for r in responses if r.success}

__init__(max_workers=None, max_per_second=inf, show_progress=False)

Initialize rate-limited executor.

Parameters:

• max_workers (int) – Maximum concurrent threads.

• max_per_second (float) – Maximum calls per second.

• show_progress (bool) – Display progress bar during execution.

execute(fn, requests, desc='Processing', unit='item')

Execute function on all requests and return responses in order.

Parameters:

• fn (Callable) – Function that takes request.item and returns a result.

• requests (list) – List of TaskRequest objects to process.

• desc (str) – Description for progress bar.

• unit (str) – Unit name for progress bar.

Returns:

List of TaskResponse objects in same order as requests.

Return type:

list[TaskResponse]

execute_items(fn, items, desc='Processing', unit='item')

Execute function on items with auto-generated request IDs.

Parameters:

• fn (Callable) – Function that takes an item and returns a result.

• items (list) – List of items to process.

• desc (str) – Description for progress bar.

• unit (str) – Unit name for progress bar.

Returns:

List of TaskResponse objects with request.id = index.

Return type:

list[TaskResponse]

submit(fn, *args, **kwargs)

Submit a callable to be executed with rate limiting.

Parameters:

• fn – Callable to execute.

• args – Positional arguments.

• kwargs – Keyword arguments.

Returns:

Future representing the result.

Return type:

Future

shutdown(wait=True, cancel_futures=False)

Shutdown the executor.

Parameters:

• wait (bool) – Wait for pending futures to complete.

• cancel_futures (bool) – Cancel pending futures.

Return type:

None