System Health, Async Queues & Infrastructure Monitoring
System Health, Async Queues & Infrastructure Monitoring
The YeneSchool platform manages high-throughput background operations across thousands of concurrent users — from bulk student report card rendering and national exam imports to real-time Telegram and SMS dispatching. The Super Admin monitoring suite provides real-time observability over these asynchronous systems.
1. BullMQ Background Worker Queues
Background operations run on Redis-backed BullMQ queues to avoid blocking user HTTP requests:
| Queue Identifier | Typical Workload | Health Indicators |
|---|---|---|
notifications | SMS alerts, Telegram messages, Web Push, and institutional emails. | Jobs/sec, failure rate, retry backlog. |
reports | Terminal report card PDF compilation, broadsheet generation. | Memory consumption per worker, average rendering time. |
bulk-upload | Parsing and validating student/staff CSV records (up to 5,000 rows). | Queue wait latency, row validation errors. |
ai-tasks | Vector embedding updates, weekly lesson plan generation. | Token consumption, provider timeouts. |
Inspecting Queue Failures
In the Super Admin console:
- Navigate to System Health > Queues.
- If a queue shows a non-zero Failed count, click View Failed Jobs.
- Inspect the stack trace (e.g., SMS Provider rate limit exceeded or Malformed phone number).
- Click Retry Job or Retry All Failed to re-dispatch idempotent tasks.
2. Database Connection Pooling (PgBouncer)
To prevent PostgreSQL connection exhaustion during high-concurrency periods (e.g., morning roll-call and term-end report card publishing):
- PgBouncer sits in front of PostgreSQL, multiplexing connections.
- Super Admins can monitor active server connections, waiting client queries, and connection pool utilization.
- If connection pools approach 90% saturation, the dashboard alerts administrators to increase
max_connectionsor scale pool workers.
3. Global Audit Logs
All sensitive Super Admin actions are permanently recorded in the immutable audit log:
- Tenant provisioning and deletion
- Subscription manual approvals and overrides
- Quota modifications
- Director password resets
Each audit log entry captures:
- Timestamp (ISO 8601 UTC)
- Actor User ID & IP Address
- Target School ID & Action Type
- Pre-change and post-change JSON state diffs
Subscriptions & Billing
Managing SaaS billing plans, manual Ethiopian bank slip review workflows, CBE and Telebirr verification, and automated renewal reminders.
Deployment Architecture
Comprehensive architecture overview of YeneSchool's production container topology — PostgreSQL with pgvector, PgBouncer, Redis BullMQ, NestJS, Next.js, and Nginx.