<div dir="ltr">So my original comment on the ion package was a bit dubious about its use on microcontrollers mainly because it converts everything to text including blobs (JSON must be pure text and cannot have binary data in it). As I said probably using base64. This would make the blob (binary) data a lot bigger and would require a base64 decoder on the MCU. I don't consider this to be a wise choice, though simple formats may work fine.<div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 22, 2021 at 3:59 PM Jon Wolfe <<a href="mailto:jonjwolfe@anibit.com">jonjwolfe@anibit.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>nanopb's readme just says it's an implementation of Protocol
Buffers for embedded use, without really describing what Protocol
Buffers is. <br>
</p>
<p>The readme for the main Protocol Buffers says it's "Google's
language-neutral,
platform-neutral, extensible mechanism for serializing structured
data"</p>
<p>In other words, a system for converting data structures to and
from a block of bytes. That description also fits Ion.</p>
<p>They just seem to differ in how they do that and in some of the
aspects of the serialized data. Ion supports text or binary format
for serialized data, and it's text format is a superset of json.
protobufs are primarily a binary format, but the original google
developed version does also support json (nanopb presently does
not). Protobufs require a well-defined schema for the data
structures, and there is tooling to process that schema into code.
Ion does not appear to require a schema (that's the 'self
describing' part of it). I'm guessing Ion has a rich set of API's
for interrogating the de-serialized data for structure. (Protobuf
has reflection API,s but they are populated based on code
generated by the tooling from the schema definition, not from the
data object itself). <br>
</p>
<p>If Amazon Ion does support embedded systems, I'd imagine it comes
with a lot of limitations on using the text formats. Parsing json
on embedded devices can be pretty expensive, memory wise. <br>
</p>
<p><br>
</p>
<div>On 11/22/2021 2:33 PM, Carl Nobile
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Jon, none of what you are talking about is in the
GitHub README file where it says it's buffering code. The
external docs, after looking at them, hardly mention buffering
at all, so there's a definite misunderstanding if one only reads
the README. They need to put a bit more info in the README.
<div><br>
<div>With 'amazon.ion' there is no encoding or decoding as it
would be in 'nanopb', you just pull data out of the
user-defined structure, maybe this is what you mean when you
say it's "self describing". JSON is just keywords and values
and/or a list of items, that's it.</div>
<div>JSON is an acronym for JavaScript Object Notation because
this is where the format was first created, however, it
almost directly matches how the internals of Python works
also, so it can be parsed with Python extremely easily.</div>
<div><br>
</div>
<div>~Carl</div>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Nov 22, 2021 at 1:56
PM Jon Wolfe <<a href="mailto:jonjwolfe@anibit.com" target="_blank">jonjwolfe@anibit.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Carl, that is not correct. I have worked with nanopb
specifically with two micros passing data encoded with
nanopb over UART and I2C. Also, I'm 99.9% certain that the
on-the-wire format used by nanopb is compatible with the
mainstream protocol buffers format used on
desktop/servers. I've used the full protocol buffers
libraries for both communication over sockets and for
serializing data structures to disk on PC. <br>
</p>
<p>nanopb does not deal directly with the transport of the
data (eg UART, I2C, or sockets), it just can convert a
data structure back and forth from a block of bytes. What
you do with that block of bytes is up to you. It's my
understanding that that is pretty much what the Amazon
library does as well. <br>
</p>
<p>The main difference from my brief reading about Ion as
that it's "self describing" where the data contains a
description of itself. Protobuf doesn't do that, but you
share ".proto" files between both sides and the protoc
compiler generates wrapper code in the language you're
using. It has capability to handle backward compatibility,
so you can modify the data structures, but both sides do
need to have some basic idea about the structure, it's not
inherent to the data stream. Proto bufs let you have a
strongly typed contract on both ends of a communication
channel, and it sounds like Amazon Ion lets you have that
contract more loosely defined. <br>
</p>
<p>The main Protocol Buffers project has a companion project
called GRPC that builds on top of Protocol Buffers and is
geared toward "client server" communications. That library
does handle the transport of data as well as the packaging
of it. It generates server and client code for you to
handle the transport. Think of that like REST, but with a
binary format and strongly typed contract. <br>
</p>
<p><br>
</p>
<div>On 11/22/2021 1:37 PM, Carl Nobile via TriEmbed wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">So the two packages mentioned in this
thread do not do the same thing and cannot replace each
other.
<div>The 'amason.ion' package is a data format structure
implemented using JSON, whereas 'nanopb' is a
buffering system specifically for microcontrollers. In
other words, 'nanopb' CANNOT be sent over a wire
protocol where amazon.ion can be.</div>
<div>Interestingly they can be used together where
amazon.ion can be buffered by 'nanopb' which may help
with larger 'amazon.ion' data packets.</div>
<div>~Carl</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Nov 22, 2021
at 11:30 AM Peter Soper via TriEmbed <<a href="mailto:triembed@triembed.org" target="_blank">triembed@triembed.org</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> <span style="font-family:monospace">Nanopb
looks way cool. Thanks!</span> <br>
</div>
_______________________________________________<br>
Triangle, NC Embedded Computing mailing list<br>
<br>
To post message: <a href="mailto:TriEmbed@triembed.org" target="_blank">TriEmbed@triembed.org</a><br>
List info: <a href="http://mail.triembed.org/mailman/listinfo/triembed_triembed.org" rel="noreferrer" target="_blank">http://mail.triembed.org/mailman/listinfo/triembed_triembed.org</a><br>
TriEmbed web site: <a href="http://TriEmbed.org" rel="noreferrer" target="_blank">http://TriEmbed.org</a><br>
To unsubscribe, click link and send a blank message:
mailto:<a href="mailto:unsubscribe-TriEmbed@bitser.net" target="_blank">unsubscribe-TriEmbed@bitser.net</a>?subject=unsubscribe<br>
<br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">--------------------------------------------------------------<br>
Carl J. Nobile (Software Engineer/API Design)<br>
<a href="mailto:carl.nobile@gmail.com" target="_blank">carl.nobile@gmail.com</a><br>
--------------------------------------------------------------</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Triangle, NC Embedded Computing mailing list
To post message: <a href="mailto:TriEmbed@triembed.org" target="_blank">TriEmbed@triembed.org</a>
List info: <a href="http://mail.triembed.org/mailman/listinfo/triembed_triembed.org" target="_blank">http://mail.triembed.org/mailman/listinfo/triembed_triembed.org</a>
TriEmbed web site: <a href="http://TriEmbed.org" target="_blank">http://TriEmbed.org</a>
To unsubscribe, click link and send a blank message: <a href="mailto:unsubscribe-TriEmbed@bitser.net?subject=unsubscribe" target="_blank">mailto:unsubscribe-TriEmbed@bitser.net?subject=unsubscribe</a>
</pre>
</blockquote>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">--------------------------------------------------------------<br>
Carl J. Nobile (Software Engineer/API Design)<br>
<a href="mailto:carl.nobile@gmail.com" target="_blank">carl.nobile@gmail.com</a><br>
--------------------------------------------------------------</div>
</div>
</blockquote>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">--------------------------------------------------------------<br>Carl J. Nobile (Software Engineer/API Design)<br><a href="mailto:carl.nobile@gmail.com" target="_blank">carl.nobile@gmail.com</a><br>--------------------------------------------------------------</div></div>