[TriEmbed] Anybody using this with C?

Mon Nov 22 14:58:53 CST 2021

nanopb's readme just says it's an implementation of Protocol Buffers for 
embedded use, without really describing what Protocol Buffers is.

The readme for the main Protocol Buffers says it's "Google's 
language-neutral, platform-neutral, extensible mechanism for serializing 
structured data"

In other words, a system for converting data structures to and from a 
block of bytes. That description also fits Ion.

They just seem to differ in how they do that and in some of the aspects 
of the serialized data. Ion supports text or binary format for 
serialized data, and it's text format is a superset of json. protobufs 
are primarily a binary format, but the original google developed version 
does also support json (nanopb presently does not). Protobufs require a 
well-defined schema for the data structures, and there is tooling to 
process that schema into code. Ion does not appear to require a schema 
(that's the 'self describing' part of it). I'm guessing Ion has a rich 
set of API's for interrogating the de-serialized data for structure. 
(Protobuf has reflection API,s but they are populated based on code 
generated by the tooling from the schema definition, not from the data 
object itself).

If Amazon Ion does support embedded systems, I'd imagine it comes with a 
lot of limitations on using the text formats. Parsing json on embedded 
devices can be pretty expensive, memory wise.

On 11/22/2021 2:33 PM, Carl Nobile wrote:
> Jon, none of what you are talking about is in the GitHub README file 
> where it says it's buffering code. The external docs, after looking at 
> them, hardly mention buffering at all, so there's a definite 
> misunderstanding if one only reads the README. They need to put a bit 
> more info in the README.
>
> With 'amazon.ion' there is no encoding or decoding as it would be in 
> 'nanopb', you just pull data out of the user-defined structure, maybe 
> this is what you mean when you say it's "self describing". JSON is 
> just keywords and values and/or a list of items, that's it.
> JSON is an acronym for JavaScript Object Notation because this is 
> where the format was first created, however, it almost directly 
> matches how the internals of Python works also, so it can be parsed 
> with Python extremely easily.
>
> ~Carl
>
>
> On Mon, Nov 22, 2021 at 1:56 PM Jon Wolfe <jonjwolfe at anibit.com> wrote:
>
>     Carl, that is not correct. I have worked with nanopb specifically
>     with two micros passing data encoded with nanopb over UART and
>     I2C. Also, I'm 99.9% certain that the on-the-wire format used by
>     nanopb is compatible with the mainstream protocol buffers format
>     used on desktop/servers. I've used the full protocol buffers
>     libraries for both communication over sockets and for serializing
>     data structures to disk on PC.
>
>     nanopb does not deal directly with the transport of the data (eg
>     UART, I2C, or sockets), it just can convert a data structure back
>     and forth from a block of bytes. What you do with that block of
>     bytes is up to you. It's my understanding that that is pretty much
>     what the Amazon library does as well.
>
>     The main difference from my brief reading about Ion as that it's
>     "self describing" where the data contains a description of itself.
>     Protobuf doesn't do that, but you share ".proto" files between
>     both sides and the protoc compiler generates wrapper code in the
>     language you're using. It has capability to handle backward
>     compatibility, so you can modify the data structures, but both
>     sides do need to have some basic idea about the structure, it's
>     not inherent to the data stream. Proto bufs let you have a
>     strongly typed contract on both ends of a communication channel,
>     and it sounds like Amazon Ion lets you have that contract more
>     loosely defined.
>
>     The main Protocol Buffers project has a companion project called
>     GRPC that builds on top of Protocol Buffers and is geared toward
>     "client server" communications. That library does handle the
>     transport of data as well as the packaging of it. It generates
>     server and client code for you to handle the transport. Think of
>     that like REST, but with a binary format and strongly typed contract.
>
>
>     On 11/22/2021 1:37 PM, Carl Nobile via TriEmbed wrote:
>>     So the two packages mentioned in this thread do not do the same
>>     thing and cannot replace each other.
>>     The 'amason.ion' package is a data format structure
>>     implemented using JSON, whereas 'nanopb' is a buffering system
>>     specifically for microcontrollers. In other words, 'nanopb'
>>     CANNOT be sent over a wire protocol where amazon.ion can be.
>>     Interestingly they can be used together where amazon.ion can be
>>     buffered by 'nanopb' which may help with larger 'amazon.ion' data
>>     packets.
>>     ~Carl
>>
>>
>>     On Mon, Nov 22, 2021 at 11:30 AM Peter Soper via TriEmbed
>>     <triembed at triembed.org> wrote:
>>
>>         Nanopb looks way cool. Thanks!
>>         _______________________________________________
>>         Triangle, NC Embedded Computing mailing list
>>
>>         To post message: TriEmbed at triembed.org
>>         List info:
>>         http://mail.triembed.org/mailman/listinfo/triembed_triembed.org
>>         TriEmbed web site: http://TriEmbed.org <http://TriEmbed.org>
>>         To unsubscribe, click link and send a blank message:
>>         mailto:unsubscribe-TriEmbed at bitser.net?subject=unsubscribe
>>
>>
>>
>>     -- 
>>     --------------------------------------------------------------
>>     Carl J. Nobile (Software Engineer/API Design)
>>     carl.nobile at gmail.com
>>     --------------------------------------------------------------
>>
>>     _______________________________________________
>>     Triangle, NC Embedded Computing mailing list
>>
>>     To post message:TriEmbed at triembed.org
>>     List info:http://mail.triembed.org/mailman/listinfo/triembed_triembed.org
>>     TriEmbed web site:http://TriEmbed.org  <http://TriEmbed.org>
>>     To unsubscribe, click link and send a blank message:mailto:unsubscribe-TriEmbed at bitser.net?subject=unsubscribe  <mailto:unsubscribe-TriEmbed at bitser.net?subject=unsubscribe>
>>
>
>
> -- 
> --------------------------------------------------------------
> Carl J. Nobile (Software Engineer/API Design)
> carl.nobile at gmail.com
> --------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.triembed.org/pipermail/triembed_triembed.org/attachments/20211122/f3473b1d/attachment.htm>