AI & real-time insights

Intelligent systems

Engineered an ultra-low latency event processing engine processing 100,000 events/sec with real-time machine learning anomaly isolation

Intelligent systems interface concept

Project context

The risk platform needed to interpret incoming activity quickly enough for a decision to remain useful Its focus was real-time fraud detection at a reported processing volume of 100,000 events per second Streaming, model inference, and delivery of the resulting decision therefore had to be considered as one connected system

The challenge

Fraud detection must identify suspicious activity while events are still relevant The challenge was combining high-volume streaming with low-latency machine learning decisions

Key priorities

  • Process high-volume event streams without losing the value of timely decisions
  • Connect anomaly detection to downstream systems with low communication overhead
  • Consider detection quality alongside processing speed

The approach

Build around a streaming pipeline

Apache Flink provides the stream-processing component of the project stack The approach processes incoming events as an ongoing flow rather than waiting for a later batch This supports the requirement to identify suspicious activity while the underlying transaction or event is still relevant

Connect machine learning to the event flow

Python-based machine learning supplies the anomaly-detection component A model’s prediction becomes useful only when it is connected to the surrounding system: incoming information must reach the model, and its output must reach the service that acts on it The project treats this connection as part of the latency requirement

Support fast communication and analysis

The supporting stack includes Kubernetes, ClickHouse, and gRPC These address workload deployment, analytical data access, and service communication respectively Their role is to support the streaming and decision path, rather than treating model inference as an isolated feature

The outcome

<12msEnd-to-End Decision Latency

Prevented $4.2M in fraudulent transactions in 90 days

100,000 Events/sec • 99.4% Model Precision

The project summary reports decision latency below 12ms, model precision of 99.4%, and $4.2 million in prevented fraudulent transactions over 90 days Precision and latency measure different things; a fast decision still needs to be useful The case illustrates the importance of connecting model quality with the engineering of the surrounding event pipeline

Let’s build your
next project

Tell us what you want to build We’ll help you define the next step

Discuss your project