Demo UTS Sistem Terdistribusi: Pub-Sub Log Aggregator + Idempotent Consumer (Exactly-Once)

⏺ Demonstrasi UTS mata kuliah Sistem Terdistribusi & Paralel: membangun
  Pub-Sub Log Aggregator dengan Idempotent Consumer dan Deduplication
  yang persisten terhadap container restart.

  Sistem menerima event lewat HTTP POST /publish, masuk ke asyncio.Queue
  in-memory, lalu diproses consumer worker yang mengecek dedup store
  berbasis SQLite (WAL mode) dengan composite key (topic, event_id).
  Pola at-least-once delivery + idempotent consumer = exactly-once
  semantics, sesuai Van Steen & Tanenbaum (2023) Bab 3, 6, dan 7.

  🧪 Yang dibuktikan di demo
  • Golden path: 3 event unik → processed: 3, duplicates: 0
  • Idempotency: evt-001 dikirim 5x → unique_processed tetap, duplicate_dropped naik 5
  • Crash recovery: container dihapus & dibuat ulang, dedup store SQLite tetap ingat
  • Throughput: ~4500 events/sec dengan 20% duplicate rate (4000 unik + 1000 duplikat)

  🛠 Tech stack
  Python 3.11 · FastAPI · uvicorn · Pydantic · asyncio.Queue ·
  SQLite (WAL) · Docker · pytest (35/35 lulus)

  📦 Repository (source code)
github.com/Faedyl/pubsub-dedup-aggregator

  📚 Referensi utama
  Van Steen, M. & Tanenbaum, A. S. (2023). Distributed Systems
  (4th ed.). Bab 2 (Architectures), Bab 3 (Processes & Communication),
  Bab 4 (Naming), Bab 6 (Coordination & Fault Tolerance),
  Bab 7 (Consistency & Replication).

  👤 Mahasiswa
  Nama : Fadhil Awalia Kusuma
  NIM  : 11231023
  Kelas: Sistem Terdistribusi & Paralel

  #SistemTerdistribusi #PubSub #Idempotency #Deduplication #FastAPI #Docker #SQLite #UTS