[TriEmbed] Update on my Stability Issue

[Josh Wyatt]

Hi Chip,

Great writeup, thank you for sharing your knowledge.

Thanks,
Josh

On Tue, Mar 14, 2017 at 10:02 PM, Chip McClelland via TriEmbed <
triembed at triembed.org> wrote:

> All,
>
> For those who missed it, I sent a note about a month ago asking for advice
> on improving the reliability of my sensors.  These sensors would go weeks
> and measure tens of thousands of cars and then lock up.  I was struggling
> to fix a problem I could not easily reproduce so I asked this group for
> ideas.
>
> Thank you for all the great advice it has been very helpful.  I think I
> have made significant progress in my testing and, this week, I have
> deployed the updates to the sensors in the park.  Given that many of you
> took the time to give me your thoughts, I wanted to share what I have
> learned.
>
> 1) A few of you suggested making the problem worse to help improve the
> reproducibility of the issue - that was key.  I was eventually able to get
> the issue to surface in under ten minutes which helped me test potential
> fixes.
> 2) My problem seemed to be related to the Arduino’s ability to wake from
> sleep and reliably access data on the i2c bus.  I was able to find the
> specific function and eventually the  i2c command - t = RTC.get(); that
> would lock up the i2c bus and the Arduino.  The incidence of this issue was
> increased significantly when the Arduino was recently brought back from
> sleep.
> 3) The solution was to start a watchdog timer prior to using this command
> and set up the Arduino to reboot if it did not complete in a reasonable
> period.  Worked fine on the Uno but this locked up the Arduino Pro Mini.
> It seemed that the Pro mini does not use an Optiboot boot loader like the
> Uno and there is a known bug
> <https://forum.arduino.cc/index.php?topic=435172.0> that prevents the
> watchdog timer from functioning correctly.  I found a new boot loader -
> MiniCore <https://github.com/MCUdude/MiniCore> that fixed the issue and
> delivered on some cool new features.
> 4) The problem was that I have a number of boards in production which do
> not have ICSP headers.  I was able to find a
> <https://www.amazon.com/gp/product/B00V2W467I/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1>great
> adapte
> <https://www.amazon.com/gp/product/B00V2W467I/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1>r  that
> let me refresh the chips in place - highly recommended.
> 5) Changing the boot loader also changed my tool chain and I ended up
> asking the embedXcode folks for help
> <https://embedxcode.freshdesk.com/support/tickets/660> and they delivered
> a new release that let me integrate the new boot loader with the board
> definition files.  If you are a Mac user and want a better IDE, I can
> highly recommend embedXcode <http://embedxcode.weebly.com/>.
>
> With these changes and some improvements to the process of waking from
> sleep from Nick Gammon and the Arduino forums,
> <http://forum.arduino.cc/index.php?topic=336789.0> I was able to put the
> device through significant
> testing without any lock ups.  I added a “reboot” counter to EEPROM so I
> can see if more changes are needed in the future.  I just finished updating
> all the sensors in the park and will see how the real world reacts to my
> efforts.
>
> I hope there are some ideas in here that might be helpful.  Please let me
> know if you have more suggestions.  I will also be looking at my power and
> signal integrity on the boards but, for now, I think I have a solution.  If
> you would like to see what the code looks like, you can find it here
> <https://github.com/chipmc/Logger-Arduino-PIR>.  Thanks again for all
> your suggestions.
>
> Chip
>
>
>
>
> _______________________________________________
> Triangle, NC Embedded Computing mailing list
> TriEmbed at triembed.org
> http://mail.triembed.org/mailman/listinfo/triembed_triembed.org
> TriEmbed web site: http://TriEmbed.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.triembed.org/pipermail/triembed_triembed.org/attachments/20170315/28bb137f/attachment.htm>