Dev.to technology 1h ago

Panduan API Rate‑Limit: Membangun Local Proxy dengan Token Bucket

Banyak layanan API menerapkan rate limit untuk melindungi server dari overload. Ketika aplikasi Anda melebihi batas, respons 429 Too Many Requests muncul dan proses terhenti. Solusi klasik adalah retry dengan exponential backoff, tetapi pendekatan itu masih membiarkan burst request sampai ke upstream.

Pendekatan yang lebih baik: local proxy yang menampung request, mengatur keluar secara ritmis, dan menyimpan cache respons. Artikel ini menunjukkan arsitektur dan implementasi minimal menggunakan Python (FastAPI + httpx).

Arsitektur Proxy Lokal

<svg width="500" height="250" xmlns="http://www.w3.org/2000/svg">
  <rect x="20" y="80" width="100" height="50" rx="5" fill="#4a90d9"/>
  <text x="70" y="110" text-anchor="middle" fill="white" font-size="14">Client</text>
  <path d="M120 105 L180 105" stroke="#666" stroke-width="2" marker-end="url(#arrow)"/>
  <rect x="180" y="50" width="140" height="110" rx="5" fill="#f5a623"/>
  <text x="250" y="75" text-anchor="middle" fill="white" font-size="14">Local Proxy</text>
  <text x="250" y="95" text-anchor="middle" fill="white" font-size="11">Token Bucket</text>
  <text x="250" y="115" text-anchor="middle" fill="white" font-size="11">Request Queue</text>
  <text x="250" y="135" text-anchor="middle" fill="white" font-size="11">Retry Manager</text>
  <text x="250" y="155" text-anchor="middle" fill="white" font-size="11">Cache Store</text>
  <path d="M320 105 L380 105" stroke="#666" stroke-width="2" marker-end="url(#arrow)"/>
  <rect x="380" y="80" width="100" height="50" rx="5" fill="#7ed321"/>
  <text x="430" y="110" text-anchor="middle" fill="white" font-size="14">Upstream API</text>
  <defs>
    <marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto" markerUnits="strokeWidth">
      <path d="M0,0 L0,6 L9,3 z" fill="#666"/>
    </marker>
  </defs>
</svg>

Penjelasan alur:

Client mengirim request ke Local Proxy (bukan langsung ke upstream).
Proxy memeriksa Token Bucket: apakah token tersedia? Jika ya, request dilepaskan ke upstream; jika tidak, request masuk Request Queue.
Retry Manager menangani respons 429 dari upstream dengan membaca header Retry-After dan menambahkan jitter.
Cache Store menyimpan respons GET yang berhasil; request identik berikutnya disajikan dari cache tanpa menghitung ke bucket.

Implementasi Minimal (FastAPI + httpx)

# proxy.py
import asyncio, time, hashlib, json
from collections import deque
from fastapi import FastAPI, Request, Response, HTTPException
import httpx

app = FastAPI()
UPSTREAM = "https://api.example.com"
RATE = 10          # request per second
BURST = 20         # bucket capacity
CACHE_TTL = 300    # seconds

class TokenBucket:
    def __init__(self, rate, burst):
        self.rate = rate
        self.burst = burst
        self.tokens = burst
        self.last = time.monotonic()
        self.lock = asyncio.Lock()
    async def take(self):
        async with self.lock:
            now = time.monotonic()
            self.tokens = min(self.burst, self.tokens + (now - self.last) * self.rate)
            self.last = now
            if self.tokens >= 1:
                self.tokens -= 1
                return True
            return False

bucket = TokenBucket(RATE, BURST)
queue = deque()
cache = {}

async def forward(req: Request):
    key = f"{req.method}:{req.url.path}:{req.query_params}"
    if req.method == "GET" and key in cache:
        cached, ts = cache[key]
        if time.time() - ts < CACHE_TTL:
            return Response(content=cached, media_type="application/json")
    async with httpx.AsyncClient(base_url=UPSTREAM, timeout=10) as client:
        while True:
            if await bucket.take():
                break
            await asyncio.sleep(0.1)
        upstream_req = client.build_request(req.method, req.url.path, params=req.query_params, headers=dict(req.headers), content=await req.body())
        resp = await client.send(upstream_req)
        if resp.status_code == 429:
            retry = int(resp.headers.get("Retry-After", "1"))
            await asyncio.sleep(retry + 0.1 * (hash(key) % 10))
            continue
        if req.method == "GET" and resp.status_code == 200:
            cache[key] = (resp.content, time.time())
        return Response(content=resp.content, status_code=resp.status_code, media_type=resp.headers.get("content-type"))

@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
async def proxy(path: str, request: Request):
    return await forward(request)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Menjalankan Proxy

pip install fastapi httpx uvicorn
uvicorn proxy:app --port 8000

Sekarang arahkan client ke http://localhost:8000/... sebagai pengganti upstream asli.

Checklist Produksi

[ ] Tambahkan metrics (Prometheus) untuk mengamati bucket & queue.
[ ] Implementasikan circuit breaker jika upstream terus gagal.
[ ] Gunakan Redis untuk cache & bucket terdistribusi.
[ ] Tambahkan authentication (API key) pada proxy.
[ ] Tulis unit & integration test untuk retry logic.

Kesimpulan

Dengan local proxy token‑bucket, aplikasi Anda tidak lagi perlu khawatir soal 429. Proxy menyerap burst, mengatur throughput, dan mengurangi beban upstream lewat caching. Pola ini cocok untuk microservices, scraper, dan integrasi pihak ketiga.

Artikel ini dipublikasikan otomatis melalui pipeline Writer → Designer → Reviewer → Publisher.

Story originally reported by Dev.to. View at Dev.to →