[TriEmbed] Anybody using this with C?

Mon Nov 22 15:14:31 CST 2021

So my original comment on the ion package was a bit dubious about its use
on microcontrollers mainly because it converts everything to text including
blobs (JSON must be pure text and cannot have binary data in it). As I said
probably using base64. This would make the blob (binary) data a lot bigger
and would require a base64 decoder on the MCU. I don't consider this to be
a wise choice, though simple formats may work fine.

On Mon, Nov 22, 2021 at 3:59 PM Jon Wolfe <jonjwolfe at anibit.com> wrote:

> nanopb's readme just says it's an implementation of Protocol Buffers for
> embedded use, without really describing what Protocol Buffers is.
>
> The readme for the main Protocol Buffers says it's "Google's
> language-neutral, platform-neutral, extensible mechanism for serializing
> structured data"
>
> In other words, a system for converting data structures to and from a
> block of bytes. That description also fits Ion.
>
> They just seem to differ in how they do that and in some of the aspects of
> the serialized data. Ion supports text or binary format for serialized
> data, and it's text format is a superset of json. protobufs are primarily a
> binary format, but the original google developed version does also support
> json (nanopb presently does not). Protobufs require a well-defined schema
> for the data structures, and there is tooling to process that schema into
> code. Ion does not appear to require a schema (that's the 'self describing'
> part of it). I'm guessing Ion has a rich set of API's for interrogating the
> de-serialized data for structure. (Protobuf has reflection API,s but they
> are populated based on code generated by the tooling from the schema
> definition, not from the data object itself).
>
> If Amazon Ion does support embedded systems, I'd imagine it comes with a
> lot of limitations on using the text formats. Parsing json on embedded
> devices can be pretty expensive, memory wise.
>
>
> On 11/22/2021 2:33 PM, Carl Nobile wrote:
>
> Jon, none of what you are talking about is in the GitHub README file where
> it says it's buffering code. The external docs, after looking at them,
> hardly mention buffering at all, so there's a definite misunderstanding if
> one only reads the README. They need to put a bit more info in the README.
>
> With 'amazon.ion' there is no encoding or decoding as it would be in
> 'nanopb', you just pull data out of the user-defined structure, maybe this
> is what you mean when you say it's "self describing". JSON is just
> keywords and values and/or a list of items, that's it.
> JSON is an acronym for JavaScript Object Notation because this is where
> the format was first created, however, it almost directly matches how the
> internals of Python works also, so it can be parsed with Python extremely
> easily.
>
> ~Carl
>
>
> On Mon, Nov 22, 2021 at 1:56 PM Jon Wolfe <jonjwolfe at anibit.com> wrote:
>
>> Carl, that is not correct. I have worked with nanopb specifically with
>> two micros passing data encoded with nanopb over UART and I2C. Also, I'm
>> 99.9% certain that the on-the-wire format used by nanopb is compatible with
>> the mainstream protocol buffers format used on desktop/servers. I've used
>> the full protocol buffers libraries for both communication over sockets and
>> for serializing data structures to disk on PC.
>>
>> nanopb does not deal directly with the transport of the data (eg UART,
>> I2C, or sockets), it just can convert a data structure back and forth from
>> a block of bytes. What you do with that block of bytes is up to you. It's
>> my understanding that that is pretty much what the Amazon library does as
>> well.
>>
>> The main difference from my brief reading about Ion as that it's "self
>> describing" where the data contains a description of itself. Protobuf
>> doesn't do that, but you share ".proto" files between both sides and the
>> protoc compiler generates wrapper code in the language you're using. It has
>> capability to handle backward compatibility, so you can modify the data
>> structures, but both sides do need to have some basic idea about the
>> structure, it's not inherent to the data stream. Proto bufs let you have a
>> strongly typed contract on both ends of a communication channel, and it
>> sounds like Amazon Ion lets you have that contract more loosely defined.
>>
>> The main Protocol Buffers project has a companion project called GRPC
>> that builds on top of Protocol Buffers and is geared toward "client server"
>> communications. That library does handle the transport of data as well as
>> the packaging of it. It generates server and client code for you to handle
>> the transport. Think of that like REST, but with a binary format and
>> strongly typed contract.
>>
>>
>> On 11/22/2021 1:37 PM, Carl Nobile via TriEmbed wrote:
>>
>> So the two packages mentioned in this thread do not do the same thing and
>> cannot replace each other.
>> The 'amason.ion' package is a data format structure implemented using
>> JSON, whereas 'nanopb' is a buffering system specifically for
>> microcontrollers. In other words, 'nanopb' CANNOT be sent over a wire
>> protocol where amazon.ion can be.
>> Interestingly they can be used together where amazon.ion can be buffered
>> by 'nanopb' which may help with larger 'amazon.ion' data packets.
>> ~Carl
>>
>>
>> On Mon, Nov 22, 2021 at 11:30 AM Peter Soper via TriEmbed <
>> triembed at triembed.org> wrote:
>>
>>> Nanopb looks way cool. Thanks!
>>> _______________________________________________
>>> Triangle, NC Embedded Computing mailing list
>>>
>>> To post message: TriEmbed at triembed.org
>>> List info:
>>> http://mail.triembed.org/mailman/listinfo/triembed_triembed.org
>>> TriEmbed web site: http://TriEmbed.org
>>> To unsubscribe, click link and send a blank message: mailto:
>>> unsubscribe-TriEmbed at bitser.net?subject=unsubscribe
>>>
>>>
>>
>> --
>> --------------------------------------------------------------
>> Carl J. Nobile (Software Engineer/API Design)
>> carl.nobile at gmail.com
>> --------------------------------------------------------------
>>
>> _______________________________________________
>> Triangle, NC Embedded Computing mailing list
>>
>> To post message: TriEmbed at triembed.org
>> List info: http://mail.triembed.org/mailman/listinfo/triembed_triembed.org
>> TriEmbed web site: http://TriEmbed.org
>> To unsubscribe, click link and send a blank message: mailto:unsubscribe-TriEmbed at bitser.net?subject=unsubscribe <unsubscribe-TriEmbed at bitser.net?subject=unsubscribe>
>>
>>
>
> --
> --------------------------------------------------------------
> Carl J. Nobile (Software Engineer/API Design)
> carl.nobile at gmail.com
> --------------------------------------------------------------
>
>

-- 
--------------------------------------------------------------
Carl J. Nobile (Software Engineer/API Design)
carl.nobile at gmail.com
--------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.triembed.org/pipermail/triembed_triembed.org/attachments/20211122/451c7db6/attachment.htm>