Protocol Buffers - is it really faster than xml?

-
It seems google is claiming their protocol buffers are faster than xml... without any proof.

Consider AsmXml, which can process xml at over 200MB/s on old machines.

The protocol buffers from google also generate wrappers for different languages, and other nice things. But for loading structures into and out of memory, xml can be very fast.

Before claiming things like that, I think proof in the form of benchmarks are needed.

I don't doubt they thought that xml was slower, since many implementations are slower. Maybe xml is slower, but there is no proof yet. Also I'm sure the other nice features of protocol buffers make them perfectly suited for their task.

Url encoding could have been used nicely too.

Comments

shubham said…
I agree with you illume that things stated without proof are not to be believed easily. Why dont you conduct some tests and experiments to compare both xml and protocol buffers to see which is faster. As for me, i havent worked on xml, but found protocol buffers very convinient. It will be pretty enlightening to see the results of such experiments.
5:06 AM
René Dudfield said…
@shubham
unfortunately I've not enough time for such science... I'll have to leave adding science to the protocol buffers claims to its authors.

It definitely would be interesting to compare the speed of AsmXml to protocol buffers.
5:26 AM
kryton said…
my first thought was it was 'lighter' on the network.
10:59 AM
David said…
The encoding is a very simple binary data format. See the documentation here: http://code.google.com/apis/protocolbuffers/docs/encoding.html

Basically, data is encoded as a tag to identify the field, followed by the data type, followed by an optional data length for fields like strings, followed by the data.

At runtime the data is represented in your program by a generated object that can parse itself with very simple logic. It just scans through the binary fields until it recognizes a tag number and then it does a very simple copy/decode and stores the value directly in a member field.

Since this is an extremely simple and easy to serialize and parse data format (it's hard to imagine it getting much simpler!), it would be very difficult for structured XML to compete speed-wise given all the tokenizing and string -> data conversions it has to do. An XML representation of the same data would also be significantly larger, which would slow down communication between machines.
11:30 AM
Mike K said…
On pure parse algorithm alone, XML needs to read look at every character to be sure it's not the start of a new tag. If you know the number of bytes in a chunk, you can omit the test and merely add the offset to the next chunk. So weather the implementation is faster or not, algorithmically it is.

Not to mention offlining the idea of schema, generating native code for you.
12:19 AM
kunal said…
Ahh if this indeed is a bad joke . Folks at google should know markup a little better . If fast as their only concern why not try binary XML .
Maybe there schema is not upto the mark .
9:04 PM
Post a Comment

Popular posts from this blog

Draft 3 of, ^Let's write a unit test!^

-
Image
So, I started writing this for people who want to 'contribute' to Community projects, and also Free Libre or Open source projects. Maybe you'd like to get involved, but are unsure of where to begin? Follow along with this tutorial, and peek at the end in the "what is a git for?" section for explanations of what some of the words mean. Draft 1, 2018/07/18 - initial draft. Draft 2, 2019/11/04 - two full unit test examples, assertions, making a pull request, use python 3 unittest substring search, "good first issue" is a thing now. Started "What is a git for? Jargon" section. Draft 3, 2020/12/12 - default branch name is main. What's first? A test is first. A unit test is a piece of code which tests one thing works well in isolation from other parts of software. In this guide, I'm going to explain how to write one using the standard python unittest module, for the pygame game library. You can apply this advice to most python pr...
Read more

Is PostgreSQL good enough?

-
Image
tldr; you can do jobs, queues, real time change feeds, time series, object store, document store, full text search with PostgreSQL. How to, pros/cons, rough performance and complexity levels are all discussed. Many sources and relevant documentation is linked to. Your database is first. But can PostgreSQL be second? Web/app projects these days often have many distributed parts. It's not uncommon for groups to use the right tool for the job. The right tools are often something like the choice below. Redis for queuing, and caching. Elastic Search for searching, and log stash. Influxdb or RRD for timeseries. S3 for an object store. PostgreSQL for relational data with constraints, and validation via schemas. Celery for job queues. Kafka for a buffer of queues or stream processing. Exception logging with PostgreSQL (perhaps using Sentry) KDB for low latency analytics on your column oriented data. Mongo/ZODB for storing documents JSON (or mangodb for /dev/null replacemen...
Read more

post modern C tooling - draft 6

-
Image
Contemporary C tooling for making higher quality C, faster or more safely. DRAFT 0 - 10/11/18,  DRAFT 1 - 9/16/19, 7:19 PM, I'm still working on this, but it's already useful and I'd like some feedback - so I decided to share it early. DRAFT 2 - 10/1/19, mostly additions to static analysis tools. DRAFT 3 - 10/4/19, updates on build systems, package management, and complexity analysis.  DRAFT 4 - 10/6/19, run time dynamic verification and instrumentation, sanitizers (asan/ubsan/etc), performance tools, static analyzers. DRAFT 5 - C interpreter(s).  DRAFT 6 - 11/6/19 , mention TermDebug vim,  more windows debugging tools, C drawing for intro. In 2001 or so people started using the phrase "Modern C++". So now that it's 2019, I guess we're in the post modern era? Anyway, this isn't a post about C++ code, but some of this information applies there too. No logo, but it's used everywhere. Welcome to the post modern era. S...
Read more