Arduino: is it only for enthusiasts and their projects?
There is an opinion that Arduino is a system exclusively for schoolchildren and their parents who want to learn how to create electronic devices for LEDs winking, measuring the temperature in the pantry, and training a simple robot on the site in front of the house while catching the enthusiastic views of neighbors. I will not play hanky-panky, it was designed just for such purpose, for training and learning. With the help of Arduino, the entry barrier into the world of electronics was significantly reduced for all the newcomers.
The Arduino creators intend to simplify the entire system: the microcontroller itself, the development environment, language and even expansion cards that can be installed into each other without taking up lengthy soldering processes. The simplest system can be assembled literally on the knee and started up in minutes. But intentional simplification does not mean that Arduino cannot be used in more serious projects rather than the automation of home plumbing via sending consumption information to the smartphone of the responsible tenant.
It goes without saying that there is quite a variety of applications with different requirements and with different demands for the resulting quality and safety. For the automation of non-critical functions or production facilities, diverse tools can be used. And that includes those initially used for training purposes. This is one of such cases that I want to share with the readers.
Before starting the evaluation of a project of creating an industrial device, which performance, stability and overall quality have full control over a piece of very expensive equipment, I want to introduce some stipulation to avoid ambiguity and misunderstanding of my arguments and prerequisites by the reader.
Arduino, as a platform in general, has made a real revolution in the world of DIY electronics. In the pre-Arduino era the embedded systems enthusiasts, of course, existed. But they stewed, for the most part, in their own delicious juices and did not move from platform to platform since the threshold to enter the new platform was essential. It was necessary to study the architecture of the microcontroller, all its hardware, communication protocols, development environment, and the framework used. All this took time. Moreover, the system architecture philosophy of different microcontrollers’ producers is often quite different. That increased the level of difficulty when someone wanted to switch from one architecture to another.
But with the arrival of Arduino, everything changed. Programming an ATmega microcontroller board became as easy as writing a console program for Windows. A simplified framework, a lightweight programming language that does not require a prior declaration of all the variables and functions, a simple development environment, and finally means for downloading compiled code onto the board via just a USB-cable, attracted millions of users. Suddenly, a whole new big market emerged which many of hardware manufacturers and later even the major embedded chip manufacturers pointed their views to.
Arduino ecosystem began to develop, grow, spread, and change. To date, Arduino is already much more than a set of LEDs, press buttons, and Arduino Uno with a USB-cable.
For me, and as a part of this article, Arduino is an ecosystem that integrates the Arduino hardware platform (including both Arduino boards themselves and the variety of sensors developed for this platform), Arduino framework, which is applicable not only for ATmega microcontrollers, and Arduino IDE as an integrated development environment. With time however any popular ecosystem is gradually growing, absorbing many related areas, and thus its boundaries are blurring. For example, the variety of shields, whether temperature sensors or relays developed for the ESP8266 platform by WeMos or SparkFun can also be considered as a part of the Arduino ecosystem, even though ESP8266 can be programmed not only with Arduino framework and Arduino IDE but also with a proprietary RTOS IDF-Style SDK with a variety of compatible IDEs.
And if the use of Arduino ecosystem as an excellent DIY environment does not cause questions and the system is truly perfectly suited for beginners (yes, you can make it even easier - but is it really necessary?), then the use of Arduino as either an ecosystem or as its individual parts for industrial-suited devices still generates a certain skepticism among specialists, especially among those respected guys who belong to so-called "old school". Below I will try to show you whether Arduino can be used as something more serious rather than simply monitoring temperature and humidity in your country house.
New horizons require new equipment
In my company called Octoglass, we are engaged in the development of electrochromic systems. Electrochromic means that when a bias is applied towards an electrochromic device, a cascade of redox reactions are provoked, through which part of the substance of the device either acquires or loses color (i.e., begins to transmit accordingly either less or more radiation of certain wavelengths). In our company, we work with the effects of electrochromism within the visual range of the electromagnetic spectrum. One of the obvious applications of this electrochromic functionality is the consequential use of electrochromic devices in structural glazing, where at a user's request glass can be darkened or brightened (so the pure purpose is the adjustable electrical tinting). The processes that occur inside an electrochromic device are quite complex, and even a superficial description of them will require another article of at least a similar size. All these double dielectric layers, electrolytes, ions, current density distribution, and voltage drops along the glass surface, temperature dependencies, etc., require complex algorithms to control the proper voltage supply voltage to “electrochromic glass” (as we call the complete electrochromic device).
A sample of an electrochromic glass we work with
Our current commercial developments operate at a DC voltage of about one volt, while the current consumption in active phases of operation can reach more than one amp per square meter of the product surface (it should be noted that by the product here and thereafter I mean a glass triplex laminate ready for installation into an insulated glass unit (IGU) or a vehicle, including an internally co-laminated electrochromic device itself with the proper electrical outputs attached and ready to connect to the power supply terminals). In addition to the low operating voltage and high current, other requirements are imposed on the electrical power supply of the electrochromic system, such as the stability of the output voltage. Our devices require voltage delta accuracy of at least 0.1 volts, with spikes of less than 0.02 volts, as well as the capability of supplying voltage in short pulses, change the phasing of the voltage, and constant monitoring of not only the voltage itself but also the current and the temperature of the device. And as a cherry on top of this cake of requirements - the generation of an ultra-low voltage of 0.5 volts is necessary.
The main applications of our products - the electrochromic glass with variable light transmission - are the means of transport: cars, water, and railway transport, etc. Provision of a stable power supplement is a true challenge according to the conditions of this application niche. The controller, as we call our voltage source as a whole, must smooth out all the peaks, surges, and harmonic vibrations coming from its power circuits as well as from the feeding voltage. In general, we managed to implement an adequate hardware device on only the fourth attempt.
A specialized company from the capital engaged in electronics development has worked on the very first version of our controller. After a three-time delay, they were finally able to produce a solution that fully met our requirements from a hardware point of view. But we were not able to adopt the developed controller. Its cost was beyond reasonable limits and this important factor that complements all the performance of the device. We did not consider such a requirement as the cost of the controller in the first version of the specification. So, all in all, this controller did not move into serial production, and it turned out to be impossible to properly adapt and simplify it.
The second attempt has been made by a private developer who promised to implement all our ideas in just a month time and at a very low cost. But by the time when the prototype has been assembled, he died unexpectedly, which led us to a real despondency. The third version, strongly simplified, we have developed inhouse. The device was based on ESP8266 chip, and we wrote a very simple firmware that implemented only the basic functions. The controller did operate and performed its functions, but we could not implement all the necessary and sometimes complex control algorithms on a lightweight platform. We lacked quite a number of required hardware functions.
For this reason, a fourth and final attempt has been made to implement all our ideas in hardware. To do that we have hired another company that was also professionally engaged into electronics development. This time however in addition to all our requirements we have also restricted the developer to the cost of the final product. And we prudently decided to write the program code ourselves. We are constantly practicing hypotheses on our computer model of an electrochromic device and later testing them in practice. And because of such activities, we make changes - and quite significant at times - to the algorithms of the controller. Electrochromic devices are a complex thing and one can perform the research on them through his whole life constantly stumbling upon something new. Therefore, changes in control algorithms are made for practically every new major release of our technology.
It took the last contractor about two months to develop the device. But in the end, we once again were not satisfied with the result. STM32F103RB microcontroller chip consumed about 70% of its calculation power exclusively for voltage stabilization (the method of hardware PWM generation by the microcontroller with duty cycle correction to stabilize the required voltage level was used). Under such conditions, there was no room for additional business logic for maintenance of control algorithms, communication with the user, flashing of speech indicators. And even a slight delay in interrupt processing can lead to the voltage crawl beyond the dangerous limit.
The proposed board design and control method have been rejected: as it turned out, the developer implementing the project had happened to get acquainted with the requirements to the final product but understood them in its own specific way. The company managed to present the second version of the device prototype after only two weeks of “all come” development, with all their brains engaged into this work. The new device has been implemented on SMT32F103RE chip and the DAC system that is on board on this new chip was used as the basis for voltage generation.
An engineering sample of our 2 channel controller. STM32 chip is on the top next to a LED. Arduino is inside.
Thanks to the built-in hardware DAC and in general a more powerful and functional microcontroller, I - as the developer of the software part - had much more resources to implement all the business logic that has been originally planned. In the new design, again as originally planned, voltage correction is required no more than once per second, and sometimes even less. The stability of the DAC signal output allows me to set the required voltage and forget about it for a while until, for example, the next cycle of its correction is necessary depending on the device temperature shift.
After the approval of the prototype, it has taken the hardware developer about two more months to bring the hardware implementation to our very strict requirements in terms of voltage generation quality. And I in the meantime had the opportunity to understand the software part, or rather to understand what to implement it on, especially when the choice for the STM32 platform is extensive.
Dispute with an old engineer
I wrote my first code using Basic back in 1989 for Commodore 64. Basic interpreter was immediately loaded as soon as the set-top box connected to a TV and equipped with an external cassette memory device has been turned out. I didn't have any documentation for Commodore 64 Basic, so I had to study it using a manual for Soviet «Agat» computer.
Returning to the STM32 platform. STMicroelectronics has spent a lot of resources on providing developers with various tools to work with their microcontrollers. On the one hand, this is good: it is possible to take the most of the specific hardware with a tool specifically designed for it. But on the other hand, switching to another platform from the same manufacturer can be difficult due to the lack of compatibility at the code level. STM32 family microcontrollers are much more complex than ATmega originally used in Arduino: only the initialization settings are ten times more complicated. So, the development tools for STM32 are obviously also more complex and diverse. You can program STM32 at least on: Low-Level API, CMSIS, HAL, Mbed, SPL, Arduino, Zephyr, and probably via a few other different ways that I’m not even aware of. And these are only frameworks. There are additionally also several integrated development environments from Kail to an online environment for Mbed.
Thanks to the presence of the prototypes of our controllers I had a unique chance to choose the most appropriate implementation tool for the task. There is no dedicated software developer in our company who could work on the development of the firmware for the controller starting in the morning and till evening; we couldn't even give this work to the outsource because the requirements have been constantly changing in accordance with the results of our R&D on the electrochromic devices themselves. And maintaining relevant specifications together with the correct management of requirements with an external company would have required more resources than developing the program code in-house. That is why I had to take over the role of the developer and combine it with my other activities.
So, my task was to create the built-in controller software for electrochromic glass in a short period of time with no extra work, using where possible the external libraries and in the most understandable format for future support.
Many of STM32 developers use HAL in conjunction with Keil IDE with the microcontroller pre-configuration in STM32Cube. This approach allows you to quickly and easily configure the chip and implement the simplest business logic. But at the very beginning of the process planning it has been clear to me that configuration and control of voltage generating is only a hundredth of the total code volume. All the rest is the business logic of the voltage management algorithms, user interaction, emergency checks, and emergency handling. Therefore, the idea of using HAL was not the most successful in terms of further development and increase of the controller's functions. Having provided myself easier work at the initial stage I could have spent much more resources at all the subsequent ones.
The next candidate was Mbed - a system actively promoted by STMicroelectronics as its own branded Arduino analogue. Mbed is positioned as a system that provides an easy entry for beginners, but at the same time has the capability to unlock all the potential inherent in STM32 chips. Mbed framework "out of the box" provides access to all the functions of the chips, from conventional I/O up to the operations with CAN-bus. In addition, Mbed supports RTOS mode, although RTOS feasibility on single-threaded processors embedded into STM32 chips remains in great doubt.
While studying the Mbed framework I’ve written trial firmware first for Nucleo 64 developing board with STM32F103RB chip and then for our controller prototypes with STM32F130RE. And I gave up using Mbed. Why? The new STMicroelectronics framework turned out to be not as good as described in promotional leaflets and on the manufacturer's website. It does work, it performs all its functions, there is a library repository, it has comprehensive documentation, there is an online editor, there is a specialized IDE for offline development and you can even use your favorite development environment such as PlatformIO. But I’ve also faced a number of significant downsides. Firstly, the Mbed framework is built in such an amazing way that it will recompile everything (the entire framework) when any changes are made to the code structure. It has been the first time in my life when I saw all the 16 cores of my PC 100% loaded for ten minutes straight while compiling the firmware for the microcontroller. And no settings, including modules disabling, could solve this problem in any IDE including STM Mbed IDE. It is not very comfortable to work in such conditions. And because of by-product constant recompilation of everything - the resulting firmware took much more resources than it should have. The second point, which seriously interfered with development, was the rawness of the framework itself. Despite the more or less detailed documentation, some tweaks did not work as described in the docs or even expected by a software developer. On many issues, I had to clarify these subtle points with more intelligent users of the system at forums and the situation was further complicated by the presence of two current versions of the Mbed framework (now they have three of them pushed to the mass) that have significant differences and are actually incompatible at the code level and libraries. And, thirdly, the set of libraries available through the branded repository was poor, to say the least, and the quality of the libraries left much to be desired.
In the end, a volitional decision has been made to start firmware development using an Arduino framework, or rather its clone for the STM32 platform. At that moment the Arduino framework for STM32 has been represented by two close sets differing mainly in kernels. The very first Arduino framework for STM32 with Maple kernel was not supported as actively as the newer implementation - STM32duino. The last became the framework that has been picked for the purpose of further development. By its structure, STM32duino is more a wrapper over standard STM32 frameworks (HAL, CMSIS) which provides compatibility of the ecosystem and primarily libraries written for Arduino with the STM32 platform. Hardware initialization occurs mainly during a function call: for example, if I need to use DAC on an output that can be used for a different purpose the output will be initialized as a DAC output the first time it is used and in no other way. Thus, the developer is relieved of the burden of initializing the chip, everything is done by the framework itself. The framework, which provides compatibility with Arduino, also opens access to a wide range of libraries written by countless users. Which is also very good. Finally, Arduino has my favorite String class which makes it easy to operate with string objects. Life is not nice without it and the usage of the standard string causes pain and existential suffering.
In general, rewriting the test program from Mbed to Arduino took about 1.5 evenings after dinner, however STM32duino framework was and still is actively developing primarily in terms of support for specific functions of STM32 chips. And while those are not implemented in the framework the developer has to "dive" into the underlying layers: HAL or CMSIS. And it has turned out to be the cornerstone of a dispute between me and another old embedded systems engineer.
My position is that even for a device that manages an expensive load but does not carry critical safety functions (such as for example a vehicle Traction Control or a series of steam line gate valves on a nuclear power plant) I would prefer the fastest and the least expensive way to write firmware. The old engineer, on the contrary, insisted that for any “serious” implementation only the most proven solutions supplied exclusively by the equipment manufacturer should be used. And the use of third-party libraries should be abandoned because even the authors of the libraries themselves are not aware of what nuances and errors may hide inside of them especially if they relate to Open Source or Public Domain.
From the old engineer's point of view, it is allowed to use only the code that you have written yourself or inside your company and which, accordingly, has passed through a sieve of internal checks and testing and meets the internal or accepted in your sector of industry quality standards. Otherwise, there is a risk of getting into an unpleasant situation where the code will contain errors and will not work at all as expected by the developer. Defending my position, I’ve argued that it is possible and necessary to conduct testing of not only your own code but also the code from a third-party supplier, both separately and as part of the entire hard- and software complex. And only in this case, you can be at least somewhat sure that the software product contains the minimum of errors.
In general, the discussion has been reduced to the fact that it is not worth using Arduino, especially if you have to “dig to a lower level” for some functions. But there were no special reasons announced why Arduino and other libraries should not be used for an industrial-grade device except for the potential presence of errors in them.
Practice of application
So STM32duino framework has been chosen for the implementation of the firmware. A framework that provides compatibility with the original Arduino, libraries developed for the Arduino ecosystem and is a wrapper for standard STM32 frameworks such as HAL and CMSIS. Why didn't I write directly on HAL or CMSIS (the old engineer suggested exactly that way)? That’s because the business logic of the firmware is much easier to implement on the Arduino framework and working with buttons, LEDs and other I/O elements is better done through ready-made libraries than reinventing the bike every time. I do not argue that those developers who write on HAL constantly and only under the same platform have already grown all the necessary reliable libraries by third-party developers or had written them independently. But I really did not want to spend resources to create something that can be used ready-made.
It was necessary to decide whether to use RTOS (it is also available for the Arduino framework) or to proceed with the usual means before starting active development. STM32F103RE microcontroller chip uses a single microprocessor Cortex-M3 which is not multithreaded. Accordingly, we can’t talk accordingly about any parallel execution of user code there. Yes, M3 uses an entire of three processing pipelines for operations, but these tricks are available only at the level of the processor itself and for their effective use I would probably have to “dive” down to Assembler. Thus, there is no special sense in using RTOS to run individual threads: the same can be implemented by easier means with significantly less resource consumption.
Visual Studio Code with PlatfromIO IDE
Since it had been originally assumed that the controller firmware would be complex, I’ve divided it into separate classes. For example, I have a separate class for user interaction - it flashes LEDs, beeps buzzer; there is a class responsible for handling button presses and encoder rotations; of course, there is a separate class that handles voltage control algorithms. And C++ is used for implementation instead of conventional C since I use an object model.
To ensure interaction between classes, a message queue was implemented through which class instances can exchange messages. When a button is clicked, a message is generated in the instance of a class that handles the pressing of buttons and it is sent to the queue. Then, a class instance interested in button click processing accepts the message from the queue and deletes it. In the main loop of Arduino, only the class instance handlers are called, and all events are processed within the class instance handlers themselves depending on the logic of their work and the specific state of the controller.
Since our controllers are versatile and potentially can and should work with any our as well as, in most cases, other parties’ electrochromic devices, they must be configured for the specific devices or their series. We customize our controllers with the glass we produce for each individual piece of it thus providing maximum safety performance and minimizing the operational degradation of electrochromic components. A shell has been written to interact with an operator, perceiving commands through a terminal connection and outputting a variety of useful information. Thanks to the advanced command language several controllers can be clustered or controlled from an external device without buttons. Configuration settings of the controller can be entered in separate commands or can be loaded with batch files, which greatly facilitates the work at streaming release. All settings related to the controller are exclusively stored in the memory in an encrypted form.
I would prefer some more points as the peculiarities of software development for the Octoglass controller. Working with ultra-low voltages ranging from about 0.3V and below is difficult and unstable. We don't need such voltages, but with the help of the controller, it is possible to achieve and use them. The maximum voltage that our controller is capable of is 4 volts; at a DAC resolution of 12 bits, the program can set the operating voltage with an accuracy of 0.97 millivolts. This high accuracy is redundant for our application. An accuracy of 100 millivolts or higher is sufficient for careful application of the controller with a maximum allowable pulsation of 20 millivolts. The hardware designer has configured the system to operate in the range from 0.5 to 4 volts according to our specifications so the filtration and stabilization subsystem at voltages below 0.3 volts does not work well. Numerous bursts and fallings appear. In principle, it is possible to can live with them if there were no voltage control feedback.
At such low voltages, a feedback loop must be used because of the significant drop in voltage on the conductors between the controller and the load. Feedback can compensate this drop and ensure the optimal operation of the electrochromic device. But at ultra-low voltages, there are problems with measuring this voltage. Periodic polling of ADC with a calculation of average value is used and repeated cycle by cycle in order to measure the voltage. But if there are many peaks and dips in the output voltage, and the signal generally is noisy, then the exact measurement of voltage is significantly hampered because the process of measurement is discrete and there is a probability that all the consequential measurements will fall either on peaks or dips, which is even more likely when there are unfiltered harmonics on the input power or other pickups exist inside the circuit. Generating ultra-low-voltage without feedback is also not very good because the physical parts on a controller board also have their own tolerances and those significantly affect the end result from one controller board to another.
A bit more on the voltage feedback: initially it has been planned to keep up with the times and use a full-fledged PID algorithm to correct the output voltage. But the practice has shown that simplification to only the P part of the algorithm is quite enough. Due to the high performance of the microcontroller, our controller smoothly reaches the required voltage and supports it with enviable accuracy when using only P-algorithm.
In addition to the use of a loop in the implementation, there is another specific feature from Arduino. I am talking about interrupt handling. Our controllers are designed for installation in cars and are mainly controlled by conventional push-buttons. A pressing of a button is detected in the program via an interrupt, which somewhat unloads the main program cycle. But with Arduino, you can only use a function without parameters when specifying an interrupt handler because initially Arduino has used C. In C++ when you use instances of classes for calling any method from an instance, a pointer to an instance of this object is passed as a hidden parameter which makes it impossible to use class methods as interrupt handlers. You overcome this by specifying interrupt handlers at the function level in the main part of the code. In our case, these handlers send messages to the message queue where it is then processed by the corresponding class instances.
Instead of a conclusion Three years after the start of the epic with the development of controllers and six months after we had begun to supply our current-version controllers to our customers I can say with all certainty that the plan has finally been implemented. We have reached the desired result, sneaking through the thicket of trials and errors, but the achieved result does not just work, it also gives pleasure from the fruit of my hands, of hands of the entire team.
A customer vehicle with our glass installed. Controlled with one of the controllers.
Small Q&A section
Was it justified to use the Arduino framework to implement the software code for a STM32 microcontroller? Wasn't it easier to implement everything using native HAL and enjoy the result as many do?
HAL would allow you to implement all the hardware features of the controller "out of the box", as they say, but the price for relief at the initial stage would be increased development time for everything else that does not directly concern hardware. After all, working with hardware in the controller is less than 1% of the code, all the rest is business code, calculations, averages, forecasting, a search of values by tables, interaction with the outside world, and so on.
Has the reliability of the program been affected by using the wrapper-framework?
There are errors always and everywhere, every developer should remember this axiom. Both individuals and giant corporations with multi-level testing and trial systems make mistakes. During the development process, I found bugs in STM32duino as well, they had been reported to the framework developers and have been fixed in the next releases. Such efficiency is very pleasing. As for reliability, a lot depends on the code structure and programming quality. I always stick to simple rules that allow not to step on the obvious rakes. I write only simple code, clear, and understandable. I try never to use "hacks" available in C/C++ that allow me to type fewer characters on the keyboard to get results. I give clear names to classes, class instances, methods, functions, and variables. I always leave comments inside the source code, so I can edit the code that has been written 3 years ago after just half an hour of re-familiarizing myself with it. I don't build up “spaghetti structures” from a sequence of calls to objects, methods, and instances, so I don't "run out" of memory, never at all in my whole experience. All the devices for which I have coded operate for months and years without crashes and hangs from the software part. The controller developed in Octoglass can also work for months, which has been successfully confirmed by tests.
Has the overall performance decreased due to the use of the framework-wrapper?
STM32 microcontrollers are very powerful and their performance is usually enough for much more serious tasks. At the same time, modern compilers perform real miracles in terms of optimization during compilation. Nevertheless, the performance issue certainly worried me as well. But all the worries were in vain: the loop time and, therefore, the reaction time on events is only 0,5 milliseconds which is more than enough for our application conditions.
What other benefits do you get from using the Arduino framework?
Versatility and compatibility with other hardware platforms. After writing the code once it will be possible to transfer it to other platforms. For example, if there is a need to migrate from STM32 to ESP32 then it will only be required to change the hardware binding in the code and replace those of the code sections which were accessing the lower layers of frameworks stack for one reason or another. With the use of macros, you can generally write one program for several platforms at a time.
To summarize all of the above I can confidently state that the use of frameworks that facilitate and accelerate development - such as Arduino - is not only possible for commercially manufactured and used devices, but also makes perfect sense in terms of resource optimization. I would nevertheless also like to hear the reader’s opinion on the matter: please share in the comments your experience in developing "serious" things with "non-serious" means.