John

Protobuf # 1. Origins & History # Protobuf was born inside Google around 2001 back when Jeff Dean and Sanjay Ghemawat were building the core infrastructure. In the beginning, it was yet another attempt to fix a recurring scale problem: thousands of servers needed to exchange structured data over the network and store it on disk, and the existing options were bad. Most nonstandard binary formats broke every time someone added a field. XML was oversized and slow to parse at the scale they needed. So they built an in-house format that was eventually named proto1. It was compact, fast, and reliably adaptable. It allowed old code to ingest new message formats and new code could read the old without anything breaking. Between 2005 and 2010 it had become the connective tissue in Google’s infrastructure — pretty much every server-to-server call and most stored data at the company was (and still is) in the protobuf format. In July 2008, Google open-sourced the cleaned-up second generation as proto2, in a release led by Kenton Varda. Proto3 followed with its stable 3.0 release in 2016, simplifying the language and allowing it to support many languages consistently — the “serialization” layer of gRPC was made available to the open-source community in 2015 as the public descendant of its internal Stubby RPC system. Currently, the “editions” models (released in 2023 & 2024) comprise the current generation. A fun genealogical footnote: Varda, the engineer who open-sourced protobuf and maintained proto2, later left and built Cap’n Proto, a competing format designed around eliminating the parse step entirely. Facebook’s Thrift was built by ex-Googlers reproducing the idea. Protobuf isn’t just a format; it’s the common ancestor of a whole family.

IPFIX vs. IE315/SPM: Flow Telemetry Architectures for DDoS Detection # 1. Framing: What DDoS Detection Demands from Telemetry # The Traditional Paradigm: Splitting the Defense # Traditionally, DDoS defense was a game of architectural division. Network operators split attacks into two categories based on how they were most efficiently managed: volumetric floods and application-layer exploits. For Layer 3 and 4 volumetric floods (like UDP amplification / reflection blasts), the primary goal was protecting core bandwidth. Every day the core performance is handicapped by a perpetual barrage of short-term, high frequency attack traffic. The collective effect of these attacks are significant, and can be difficult to effectively/efficiently count. On a particularly bad day, a large flood carried into the core will actually overwhelm interface links between routers. For enterprise this translates to lost availability, while for service providers this could break tier one services like DNS, or simply impact customer experience. Consequently, these attacks were detected and policed at the outermost network edge. Operators monitored basic NetFlow metadata from edge routers to spot massive packet-per-second spikes, then utilized high-throughput ACLs or BGP routing redirections to drop or scrub the raw volume before it could enter the core network. Conversely, Layer 7 application attacks required an entirely different philosophy. Because these threats mimic legitimate user traffic (like malicious HTTP GET floods), standard edge routers could not see them. Efficiency demanded moving the defense inline, right at the point of convergence closest to the target application. This allowed an inline Web Application Firewall (WAF) or local application proxy to step in. Because it sat directly in the traffic path, the WAF possessed the cryptographic and processing capacity to decrypt TLS traffic, inspect deep packet headers, analyze cookies, and issue JavaScript challenges to weed out bots from human users.