Variadic Print::print

148 views
Skip to first unread message

cousteaulecommandant

unread,
Jan 3, 2017, 12:23:02 PM1/3/17
to Developers
I have been working on a variadic implementation of Print::print[ln] so that Serial.print[ln] can be called with multiple arguments at once.  I have tested it and it seems to work well.  My plan is to make a pull request with all this but I wanted to discuss it first.
Here's a summary of features:
  • print() can be called with multiple arguments, e.g. print("Name=", name, " Age=", 42); will be equivalent to print("Name="); print(name); print(" Age="); print(42);
  • println() can be called with multiple arguments, which will be printed as a single line; e.g. println("Name=", name, " Age=", 42); will be equivalent to print("Name="); print(name); print(" Age="); println(42);
  • Both functions return the total number of bytes written (the sum of all subcalls).
  • The implementation is backwards compatible.  Calls such as print(num, HEX) are still supported.  print("0x", 42, 16, "=", 42) will print "0x2A=42".  This is, two numbers in a row are interpreted as (number, base) (or (number, digits) for doubles).  Numbers may still be concatenated like print(n1, "", n2) (although I don't think this would be too useful anyway).  (Personally I don't quite like this convention and think a better way to do this would be with print(String(num, HEX)), but I think backwards compatibility is important.  In a future, it would be nice if some sort of formatters are implemented (and in that case I'd pass them as the first argument).)
  • The templates are inlined, using not only the inline keyword (which was implicit anyway) but also GCC's __attribute__ ((always_inline)).  This makes the variadic call identical to multiple 1-argument calls, so the resulting binaries should be identical to that (otherwise this might create a new function for each way print() is called, which seems a bit crazy).
  • This implementation makes heavy use of C++11 variadic templates, so C++11 support is a must.  (This could be guarded in #if __cplusplus>=201103L, just in case; but I don't think it's necessary as it overcomplicates things.)

There are other features that I want to discuss (I've already implemented them; it's just a matter of whether or not to remove them):

  • Inline print for integers (INLINE_PRINT_):  this replaces the calls to print([un]signed char/short/int) with inline calls to print([un]signed long). This reduces the binary size for programs that print both ints and longs, for example; and also simplifies the code in Print.cpp.  Additionally, it adds explicit support for signed char (≠char), [un]signed short, and float.  I ran some tests on some random sketches I found on internet and in all cases the executable size was slightly reduced or equal.
  • Inline replace println(...) with print(...); println() (INLINE_PRINTLN_):  this way, println("Age=", 42) becomes print("Age="); print(42); println().  This simplifies Print.cpp a lot, which is good from a maintainability point of view, and potentially reduces the executable size, as it removes a lot of functions from Print.cpp.  (The way it is implemented depends on C++11 variadic templates, but I think it can be changed so that it doesn't.)
  • C's printf-like Print::printf() method (SERIAL_PRINTF_):  this method uses <cstdio>'s vsnprintf() to print the result to a local buffer, and then write that.  This is a rather huge function that uses about 1.5 KB of program memory when used, plus it doesn't support Arduino Strings (unless you use the .c_str() method) so it probably won't be too practical, but I guess people used to C may like it.
  • Serial<< operator (SERIAL_SHIFT_OP_):  this operator allows doing Serial << "Age=" << 42; but it doesn't allow "formatters" of any kind (HEX, digits...).  It's there for people used to C++'s cout.  There's no `endl` so far; one would have to do <<"\r\n" explicitly for that.

Attached you'll find the Print.h file, which has macro definitions at the beginning that can be commented out to disable individual features; as well as the Print.cpp file (for AVR, not SAM).  These macros and all the #ifdef would be removed on the final code I put on the PR.


What do you think?  Any opinions on which features should or shouldn't go here, or any other comment?  I think this could improve the user experience considerably.

Print.h
Print.cpp

Adrian Godwin

unread,
Jan 3, 2017, 12:33:21 PM1/3/17
to devel...@arduino.cc
Is it possible to do number formatting such as zero or space padding ?
Although the existing syntax is painful, its worst feature is the inability to print columns of aligned values.
For this reason, I always use printf if I can afford the memory.


--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

cousteaulecommandant

unread,
Jan 3, 2017, 2:28:39 PM1/3/17
to Developers
On Tuesday, January 3, 2017 at 6:33:21 PM UTC+1, Adrian Godwin wrote:
Is it possible to do number formatting such as zero or space padding ?
Although the existing syntax is painful, its worst feature is the inability to print columns of aligned values.
For this reason, I always use printf if I can afford the memory.

Not with print/println; that would involve adding extra functionality and I wanted to keep this as simple as possible.  However, the Serial.printf method I proposed would do it: Serial.printf("%08X %8d", n1, n2) (but as you point out using this method creates a large executable, so it should be used with caution).

I agree that the current formatting syntax is rather messy anyway, so I don't think adding extra formatters to Print.cpp is the way to go.  Instead, I think formatters should be applied to arguments before going to print(), something like Serial.print(HEX(n)) rather than the current Serial.print(n, HEX).  (Here, HEX() would be a function returning a String.)  Many formatting functions could be added to WString.cpp for this, and Print.cpp could be stripped from all the redundant formatting functions.
Another option would be to make HEX(n) et al return some sort of "number with format" class, and let Serial.print deal with that, but it feels too complicated IMO.
In any case, this would fall in the "future work" category.

Andrew Kroll

unread,
Jan 3, 2017, 2:46:06 PM1/3/17
to devel...@arduino.cc
I use printf 99.9% of the time.
I  never really have been ever that cramped for flash or RAM space anyway. Sometimes the MCU doesn't fit the task very well as far as resources, or it may just fit, and requires tricks.

That said... here are my tips, and they probably should be placed someplace as a handy guide, with perhaps a demo that shows what the effects are.

If you are that cramped for code space, it is usually an indication that either your code is poorly written or using too many inline functions, or possibly that you just need a device with more flash. Passing pointers to data structures instead of passing a bazillion arguments helps a lot with code size and makes code easier to read too.

If you are cramped for RAM, there are a few optimizations that I use regularly. These include NOT using global variables to store anything temporary, especially if it is some large structure. This is usually one of the largest mistakes made. If you can dispense with data, your best bets are to either just use the stack, by declaring it in the method or function or use of malloc/free/new/delete (depending on personal taste). It also helps to pass pointers to a structure of data instead of passing many items, which can also help minimize flash usage. Sometimes you do need an MCU with larger RAM, but if you rethink the actual code and dispense with data once you are done with it you would be very surprised to find out how much more you can get out of a small amount of RAM, and a lot of the time you end up with faster code too.




--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.



--
Visit my github for awesome Arduino code @ https://github.com/xxxajk

cousteaulecommandant

unread,
Jan 3, 2017, 3:41:45 PM1/3/17
to Developers
On Tuesday, January 3, 2017 at 8:46:06 PM UTC+1, Andrew Kroll wrote:
I use printf 99.9% of the time.

So... that's a yes to the SERIAL_PRINTF_ feature?  :)  (If I understand correctly printf() doesn't work out of the box and requires some hacking; having a Serial.printf() method that deals with all the required calls to vsnprintf() would be useful)

Tom Igoe

unread,
Jan 3, 2017, 3:57:19 PM1/3/17
to devel...@arduino.cc
From a beginner’s point of view, I think the most valuable addition to print and println would be a Java/JavaScript-style ability to string multiple substrings and variables together with +.  e.g.:


int SensorOne = analogRead(A0);
int sensorTwo = analogRead(A1);
Serial.println(“sensor 1: “ + sensorOne + “\tsensor 2: “ + sensorTwo);

or

int red = analogRead(A0)/4;
int green = analogRead(A1)/4;
int blue = analogRead(A2)/4;
Serial.println(“<body bgcolor=\”#” + hex(red) + hex(green) + hex(blue) + “\”>);

I’ve taught beginners in different programming languages, and of all the ones I’ve dealt with, that style is the one that comes most naturally to novices. Adding that without adding significantly to the memory footprint of the functions would be the biggest improvement. It’s the one thing about print/println that I get asked for most often. 




--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

Andrew Kroll

unread,
Jan 3, 2017, 4:00:49 PM1/3/17
to devel...@arduino.cc
I wrote a library for that (a header, actually in the current project -- to be moved to a library), which covers all the major MCU brands.
With the replacement memory management in the project, printf/free/malloc/new/delete to be used from an ISR. This is very handy for debugging an ISR. I plan on making this into a library as well.

Andrew Kroll

unread,
Jan 3, 2017, 4:14:23 PM1/3/17
to devel...@arduino.cc

On Tue, Jan 3, 2017 at 3:57 PM, Tom Igoe <t.i...@arduino.cc> wrote:
From a beginner’s point of view, I think the most valuable addition to print and println would be a Java/JavaScript-style ability to string multiple substrings and variables together with +.  e.g.:


int SensorOne = analogRead(A0);
int sensorTwo = analogRead(A1);
Serial.println(“sensor 1: “ + sensorOne + “\tsensor 2: “ + sensorTwo);

or

int red = analogRead(A0)/4;
int green = analogRead(A1)/4;
int blue = analogRead(A2)/4;
Serial.println(“<body bgcolor=\”#” + hex(red) + hex(green) + hex(blue) + “\”>);

I’ve taught beginners in different programming languages, and of all the ones I’ve dealt with, that style is the one that comes most naturally to novices. Adding that without adding significantly to the memory footprint of the functions would be the biggest improvement. It’s the one thing about print/println that I get asked for most often. 

While readable, it still lacks formatting.

What if I want hex digits to always have leading zeros?
I ALWAYS want leading zeros for hex, otherwise output is horrible to look at when you have a lot of output. Others may disagree, and they are fully welcome to do so, in which case I agree to disagree with them.

What if I want leading and trailing zeros for a float?

Nobody ever considers actual formatting, and it is VERY annoying, especially if you want to parse such data in an easy way visually, or with another MCU. For example it is MUCH easier to always expect that if you are using 16bits. "0000" in a data stream is easier to deal with than parsing, looking for a separator, such as space, or '\r' or '\n', or worse yet "\r\n". This ends up eating up too more CPU time on the receiver and flash and RAM than I am willing to use, or have to write code for.
Honestly, though, when going MCU <-> MCU I usually just use straight binary with a leading tag character that lets me know how large the incoming information is, then use that to just read in how many bytes that make up that value. but for a human, you totally want these features.

Tom Igoe

unread,
Jan 3, 2017, 4:27:51 PM1/3/17
to devel...@arduino.cc
 I agree with you about formatting, but I think that printf and sprintf are far too terse for beginners to understand. And my experience has been that people use what they understand, and misuse what they don’t. 

Formatting tools do not have to be terse. I also agree that the current scheme of having to do the following is tedious:

Serial.print(“label “ );
Serial.print(variable);
Serial.print(“\tother label “);
Serial.println(otherVariable);

It was a compromise made early on to save memory, and I’ve always wanted to see an improvement over both it and printf/sprintf.  

I would prefer a third way.  Processing has some good tools for formatting, IMO. I’d encourage those.

println(hex(foo, 2)); // ensures 2-character numbers.

See:


etc.

Tom


Adrian Godwin

unread,
Jan 3, 2017, 5:04:09 PM1/3/17
to devel...@arduino.cc
IMHO the huge advantage of printf formatting is that the string looks somewhat like the thing you're trying to print. A long list of individual function calls for each element is no help at all.


To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

cousteaulecommandant

unread,
Jan 3, 2017, 7:20:15 PM1/3/17
to Developers
I have to point out that, as can be seen in avr-gcc documentation for *printf, the implementation is rather weak: it doesn't handle %f specifiers the way Arduino uses it (displays ? instead) and it lacks other printf features such as *, plus some minor things (e.g. printf("%hhX", -1) prints FFFF; it should print FF).  So it's not like printf will be a full replacement for print, and C users may find it's not what they're used to.  But it still works for simple things such as integer formatting.

I also have to add that I've found and fixed a couple of bugs in my code (I managed to get an error when trying to print a StringSumHelper, plus some mistakes with va_list).  Also, some discussion has been going on in #arduino at freenode on whether I should use malloc instead of a variable length array for Serial.printf (and whether it should return int instead of size_t).


On Tuesday, January 3, 2017 at 9:57:19 PM UTC+1, t.igoe wrote:
From a beginner’s point of view, I think the most valuable addition to print and println would be a Java/JavaScript-style ability to string multiple substrings and variables together with +.  e.g.:
Serial.println("sensor 1: " + sensorOne + "\tsensor 2: " + sensorTwo);

Well, but you can't add string literals like that, right?  (Or can you overload the char*::operator+()?)  You would need something like
    Serial.println(String("sensor 1: ") + sensorOne + "\tsensor 2: " + sensorTwo);
Alas, that looks a bit ugly.  Maybe it'd look cooler if String("...") could be abbreviated as "..."S or "..."_S, so that an S or _S suffix on a string literal means "This is an Arduino string".
This can be achieved with the magic of C++11 user-defined literals:
    String operator ""S (const char *s, size_t) {return String(s);}
    Serial.println("sensor 1: "S + sensorOne + "\tsensor 2: " + sensorTwo);
(technically suffixes not beginning with _ are reserved so you get a warning if you try to use S, but I think "sensor 1:"_S looks a bit ugly).

Again, this sounds like features for String rather than Print.  Maybe the two are more related than I thought.

Andrew Kroll

unread,
Jan 3, 2017, 8:13:17 PM1/3/17
to devel...@arduino.cc
On Tue, Jan 3, 2017 at 7:20 PM, cousteaulecommandant <cousteaule...@gmail.com> wrote:
I have to point out that, as can be seen in avr-gcc documentation for *printf, the implementation is rather weak: it doesn't handle %f specifiers the way Arduino uses it (displays ? instead) and it lacks other printf features such as *, plus some minor things (e.g. printf("%hhX", -1) prints FFFF; it should print FF).  So it's not like printf will be a full replacement for print, and C users may find it's not what they're used to.  But it still works for simple things such as integer formatting.

The only feature I have found lacking in printf was the ability to print a long long, which is 64bit, but if I slice it into two chunks it works, even if it is ugly as heck! Take a look here:
https://github.com/felis/UHS30/blob/master/libraries/UHS_FS/examples/USB_HOST_SHIELD/UHS_FS_NEW_DEMO/UHS_FS_NEW_DEMO.ino#L200-L206

Most other features would not be really needed.

Furthermore as far as printf "needing malloc", that is pure nonsense.
Back 25 years ago on Z80 printf() would just putchar() each character as it did the work, instead of building a string first.  In many older C compilers across all platforms, that is how it is done. As far as the related sprintf(), there were a few ways to handle that. the best ones would pass a pointer to the callback function that would either store in RAM the character, and a pointer to a byte in memory that would hold the character. That way it did not matter what the output actually was, and printf()/sprintf()/fprintf() and any other *printf (Yes, there are more!) would use the same format string parser.


Before you flip out and argue about too many of the finer points of the current implementation, please read on.


While it may be better on a desktop that DMAs a malloc()'d arena to output buffers and transfers on an ISR or a polled scheduler, it actually does not really matter on an MCU that is printing along on hardware serial that is going to block anyway. It has to copy each byte to the serial buffer when serial is busy. Even at 1 or 2M baud the call overhead won't make a difference. You still have to copy each byte to the serial buffer. Even virtual serial on USB won't matter, since most of them have built-in delays in order to form fuller packets to the host.

As an additional bonus if putchar() or write, or anything else was used via a callback as I suggest, then printf() "internals" end up ISR safe ***automagically*** ;-)

The real problem as I see it is that the printf family of functions was developed for a modern system, and then people expected too much from it.  It would not fit into memory, so then features got ripped out, along with a pile of other bad compromises, instead of taking lessons from the past. Writing a classic printf is *EASY*, and I would know, since I have actually done exactly this in the distant foggy past. I actually think I got the code on 8inch floppy (yes really!) and can easily get it on to a PC if anyone would like to have a look, since my old ancient 4MHz machines are actually IPv4 capable over SLIP and a null modem.



cousteaulecommandant

unread,
Jan 4, 2017, 3:15:16 PM1/4/17
to Developers
I used vsnprintf rather than hacking fprintf mostly because it was the simplest thing that came to mind and because I had no idea how to do the latter (or that it was even possible). I see that avr-gcc implements vsnprintf through fprintf so it must be easy (no idea for other architectures such as SAM) and it seems indeed more efficient (although it clashes with the idea of "small change").

I think having hex(num, [width]) and base(num, radix, [width]) would be the way to go.
Then again, that doesn't give much freedom specifying arbitrary formatters; another option is stringing formatters like hex(num).width(8).digits(5).withSign() (sadly C++ doesn't allow the optional argument syntax other languages such as Python do), or pushing them as dummy arguments that modify how Serial behaves (kinda like std::cout) but that option feels weird.

A (probably inefficient) idea for a format function:
format(num, "width", 3, "base", 16)
(returns num formatted as a String in base 16 padded to 3 digits)

cousteaulecommandant

unread,
Jan 4, 2017, 7:30:50 PM1/4/17
to Developers
Random idea for cramming width and other formatting parameters into the `base` argument, thus keeping the current structure of formatting without requiring major changes:

#define DIGITS <<8 // minimum width (chars; not necessarily digits)
#define ZEROS 0x00 // pad with zeros (default)
#define SPACES 0x80 // pad with spaces
#define SIGN 0x40 // always +/- sign
#define HEX 16 // base 16, etc

Serial.print(n, HEX | ZEROS | 6 DIGITS);

This allows combining base (6 bits), '0' flag (1 bit), '+' flag (1 bit), and width (8 bits) into a single 16-bit int field. For floats/doubles, base is replaced by decimals (up to 63, might be a bit short).

(This doesn't solve the problem of print(n, attributes) being a bit weird combined with other print(string1, string2) calls which just print 2 things, though.)

Andrew Kroll

unread,
Jan 4, 2017, 11:41:59 PM1/4/17
to devel...@arduino.cc
I'm not sorry for any rant that may appear in this email. Many people will see some of this as exactly that. If you think it is, then that is on you. I'm actually not ranting, just basing everything on pure facts and decades of experience with multiple languages.

My last two cents on this.... printf is probabbly the best way, period. Just provide new users to a cheat-sheet (heck, build it into the IDE) or else things start looking ugly and can be counter productive. Right now there are two main ways to sort-of do printf-like prints:

Using a variable:
string foo;
Serial.print(foo=....);

Or an anonymous pointer (lambda):
Serial.print((string)....);

However you have to make sure that everything you place in there is converted to a string, which is considered "bad" by people who want to:
  1. Avoid malloc/free/new/delete
  2. Have small amounts of RAM and or flash
  3. Want the code to be readable
  4. Are cramped for line space and do stupid things like use non-standard tab sizes to 'gain line room' (Yes, any tab size other than 8 IS WRONG and HARDER TO READ! Do not argue, I won't listen and I have decades of proof you are incorrect!)
Briefly, just rewrite the core output formatting parser as previously described by me (which uses a get and store callback) and everyone would jump on it, since it could avoid malloc/free/new/delete, thus not eating too much RAM or flash, and be readable by anyone who has either put in the effort to actually learn a few basic formatting anchors, or has a cheat-sheet. Oh yeah, and we won't worry about writing a line of code that scrolls on to what seems to be infinity, or got split to 10 dozen lines and is using some strange tab size that isn't used on any normal computer unless you are writing a love letter to your compiler in a wysiwyg editor.





--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

cousteau

unread,
Jan 5, 2017, 7:28:55 AM1/5/17
to Developers
First of all, let's separate the problems of /implementation/ and /functionality/ (both on topic in this discussion, but separate problems nonetheless).
I proposed a Serial.printf() function, defined its functionality/usage (same as C's printf), and provided an implementation (using vsnprintf). Providing an alternative implementation would be an enhancement over what I already proposed that wouldn't affect the functionality; personally I'm totally OK with that (I don't really care how big Print.cpp turns), it's just that I had no idea how to do it.

Some pros and cons of printf (functionality):
Pros:
- It's existed for decades, so it's well known and broadly documented.
- C programmers are used to it.
- It's already there; there's no need to reimplement it (unless there's a good reason for it such as efficiency).
- It provides plenty of formatting features in a compact manner.
- It teaches how to use printf (although Arduino is more about teaching programming than C specifically).
Cons:
- It doesn't allow printing non-POD objects such as Arduino Strings directly (a solution for this might be to add a variadic template wrapper that adds .c_str() before calling printf).
- Its syntax looks rather weird to new users.
- It requires a format string; you can't print a variable directly as Serial.print allows.
- It doesn't allow printing binary or arbitrary bases.
- avr-gcc's implementation of printf is limited (small too, nevertheless).

Conclusion: Serial.printf would be a good thing to have because of formatting, but Serial.print is more convenient from an ease of use point of view (specially if the variadic thing I proposed is implemented), therefore both methods should coexist.
I think the way to go is to implement Serial.printf (in an efficient way, if you want) AND improve Serial.print (I may be biased but I think my idea of including formatting flags into the `base` parameter may be good).

By the way, if creating custom file descriptors is as easy as you say, it may be a good idea to make a Print::operator FILE*() method so that you can just call fprintf(Serial, ...), fputs(str, Serial), etc.

Andrew Kroll

unread,
Jan 5, 2017, 8:37:53 AM1/5/17
to devel...@arduino.cc
Excellent.

Yes, there is a way to have a much better fully functioning printf().
Yes it can take file pointers.It isn't just "printf" but rather an entire family of "printf" to different objects, but they boil down to two areas:
files, and memory.

As far as printing base2, some implementations can do that, but it is best to not do something like this.

There are also many tricks that exist in the far reaches of a full printf too, most of them are rarely used.
Finally, you have to consider what standard you want (which C++ inherits). C89? C90? C99?

Again, if I find some time this weekend, I can dig up my 20+ year old Z80 code for this if you desire one that is efficient in the use of RAM and flash. You can then take that code and do as you wish. If it becomes part of AVR, that's great. If not, well, then you can at least have a really nice code sample to study and perhaps learn a few new tricks that could be of use to you in the future.



cousteau

unread,
Jan 6, 2017, 7:05:26 PM1/6/17
to Developers
Personally I wouldn't bother on reimplementing printf unless there's a need for it; I think the reduced version provided by avr-gcc is enough for most cases.

What I think is that implementing it properly using callback functions and custom FILE objects would be important.
I think I'm going to drop that feature from this "project" and move it to a separate one; it deserves its own PR and is independent from the variadic template thing anyway (both can be implemented as separate PRs).

I'm also going to forget about adding formatting capabilities to the `base` parameter *for now* for a similar reason; it doesn't affect the rest of the idea and can be implemented separately. (Also it would require modifying WString.cpp as well.) But I still think it should be done, since there seems to be quite some interest in adding extra formatting capabilities to Serial.print.

cousteau

unread,
Jan 9, 2017, 7:21:36 PM1/9/17
to Developers
OK, it has been done; thanks everybody for the feedback!  Here's the pull request:
https://github.com/arduino/Arduino/pull/5829

I've finally opted to remove all formatting and file-like features since they didn't belong in this PR and just implement the variadic feature; nevertheless it is clear that something must be done about formatting as many people seem to miss it.

Paul Stoffregen

unread,
Jan 20, 2017, 8:45:30 AM1/20/17
to devel...@arduino.cc
On 01/03/2017 12:57 PM, Tom Igoe wrote:
> From a beginner’s point of view, I think the most valuable addition to
> print and println would be a Java/JavaScript-style ability to string
> multiple substrings and variables together with +. e.g.:
>
>
> int SensorOne = analogRead(A0);
> int sensorTwo = analogRead(A1);
> Serial.println(“sensor 1: “ + sensorOne + “\tsensor 2: “ + sensorTwo);

Does anyone know a way to tell the compiler to automatically convert
"sensor 1:" to String class when used in this context?

In other words, the idea would be to get this:

Serial.println(String("sensor 1: ") + sensorOne)

But the user doesn't know to surround their string literal with
String(). As a result, the compiler treats the string as a pointer to
an array of characters and tries to add the integer to the pointer, not
what any novice would ever expect!

I've always assumed this was impossible, that it simply can't be done
with C++. Would love to be proven wrong.

Tom Igoe

unread,
Jan 20, 2017, 8:48:40 AM1/20/17
to devel...@arduino.cc
I don’t know a way, Paul, and I’ve assumed there wasn’t a known way to do it. Every time I assume something’s impossible, someone proves me wrong, though, so I’m hoping the same happens again. Thanks for asking the question.

Javier Mora

unread,
Jan 20, 2017, 2:09:45 PM1/20/17
to devel...@arduino.cc
As far as I could find out (I'm not a native C++ speaker), it's not
possible to overload operator + for two non-class types, so something
like `int number = 42; Serial.println("number=" + number);` won't be
possible. That "number=" string should be some special type. It is
also not possible to instruct C++ that string literals be of a type
other than const char[].

One way to do this is to explicitly do String("number=") or "number="_s
(the latter would imply creating an operator ""_s for strings; this is a
C++11 feature). If this would bother users too much, the only
alternative I can think of is to preprocess the .ino files and convert
all instances of "..." to String("..."), or maybe only when they're used
in an addition. This sounds like something absurdly complicated to get
right, so I'd go the first way.

In short, for concatenating and printing several arguments on a single
statement I think the only sane option is the variadic Serial.print.

Bill Perry

unread,
Jan 26, 2017, 8:02:50 PM1/26/17
to Developers
You guys should have a look at the PrintEx library: https://github.com/Chris--A/PrintEx
It is already available in the IDE library manager.
It can wrap an existing Print/Printable class object to add lots of good printf type functionality to the object including a method called printf()
It isn't as large as a full printf() because it re-uses the Print class code for some of its formatting. It also supports floats all using the standard printf format strings.
I wish the code had better examples and better documentation as some of the features/capabilities are not really documented.
But the library is very cool and easy to use and requires no changes to any of the existing Print class code.


Gabriel Staples

unread,
Feb 28, 2019, 2:29:43 PM2/28/19
to devel...@arduino.cc
Forgive me for waking up an old thread, but my plan for Arduino next time I want to use printf is to just use my own implementation through `vsnprintf()` and a statically-allocated buffer, following my pattern I outline here: https://stackoverflow.com/a/54916870/4561887. For anyone else wanting printf you might find this useful too. 

Whether or not this makes it into the Arduino source code, I guess I'm not too concerned, since I can always just drop this function into any Arduino project I write. However, something I think should probably be in the Arduino source code is a non-blocking UART Tx which sends a new char using the TX Complete Interrupt instead of doing blocking/polling uart transmits only. Then, when blocking is required, call `Serial.flush()` to block until the Tx buffer is empty.  I'm sure this could be integrated into the current source code to guarantee backwards compatibility with the current blocking transmit behavior. 

Thanks!

Gabriel Staples

Check out my Computa Pranksta USB device sold on Amazon!



Paul Stoffregen

unread,
Feb 28, 2019, 2:51:52 PM2/28/19
to devel...@arduino.cc
On 2/28/19 11:29 AM, Gabriel Staples wrote:
> However, something I think should probably be in the Arduino source
> code is a non-blocking UART Tx which sends a new char using the TX
> Complete Interrupt instead of doing blocking/polling uart transmits
> only. Then, when blocking is required, call `Serial.flush()` to block
> until the Tx buffer is empty.  I'm sure this could be integrated into
> the current source code to guarantee backwards compatibility with the
> current blocking transmit behavior.

This sounds as if you believe Serial.write() and Serial.print() are
always blocking, only returning when all data is transmitted.  Long ago
they were implemented this way (actually returning when the last 2 bytes
were in the UART's transmit registers).  But that's not how they work on
all modern versions of Arduino.

Serial.write(), which puts your data into a buffer and returns quickly. 
It's non-blocking when the buffer has enough room for your message. 
Only when the buffer is full does it block.  As soon as the buffer has
enough space, it returns quickly and the remainder of your message is
indeed transmitted from the buffer, using interrupts.

Serial.flush() does indeed block until data stored in the transmit
buffer is empty, and on most boards it does wait until the final
transmit complete interrupt.

Maybe I've just misunderstood what you're proposing?  If this is some
sort of improvement over the existing interrupt-based approach, maybe
you could explain in more detail?


Gabriel Staples

unread,
Feb 28, 2019, 3:30:00 PM2/28/19
to devel...@arduino.cc
Thanks, Paul. No, I am mistaken on that part. Not sure what I was thinking. 

From "Arduino/Source/Arduino/hardware/arduino/avr/cores/arduino/HardwareSerial.h" I just found:

#define SERIAL_TX_BUFFER_SIZE 64
and
#define SERIAL_RX_BUFFER_SIZE 64

and 
    unsigned char _rx_buffer[SERIAL_RX_BUFFER_SIZE];
    unsigned char _tx_buffer[SERIAL_TX_BUFFER_SIZE];

and I see that all print calls ultimately lead to a call to write(uint8_t), which is definitely buffered output just like you said:

"Arduino/Source/Arduino/hardware/arduino/avr/cores/arduino/HardwareSerial.cpp"
```
size_t HardwareSerial::write(uint8_t c)
{
  _written = true;
  // If the buffer and the data register is empty, just write the byte
  // to the data register and be done. This shortcut helps
  // significantly improve the effective datarate at high (>
  // 500kbit/s) bitrates, where interrupt overhead becomes a slowdown.
  if (_tx_buffer_head == _tx_buffer_tail && bit_is_set(*_ucsra, UDRE0)) {
    *_udr = c;
    sbi(*_ucsra, TXC0);
    return 1;
  }
  tx_buffer_index_t i = (_tx_buffer_head + 1) % SERIAL_TX_BUFFER_SIZE;
  // If the output buffer is full, there's nothing for it other than to 
  // wait for the interrupt handler to empty it a bit
  while (i == _tx_buffer_tail) {
    if (bit_is_clear(SREG, SREG_I)) {
      // Interrupts are disabled, so we'll have to poll the data
      // register empty flag ourselves. If it is set, pretend an
      // interrupt has happened and call the handler to free up
      // space for us.
      if(bit_is_set(*_ucsra, UDRE0))
_tx_udr_empty_irq();
    } else {
      // nop, the interrupt handler will free up space for us
    }
  }

  _tx_buffer[_tx_buffer_head] = c;
  _tx_buffer_head = i;
  sbi(*_ucsrb, UDRIE0);
  
  return 1;
}
```

Thanks!

Gabriel Staples

Check out my Computa Pranksta USB device sold on Amazon!


Bill Perry

unread,
Feb 28, 2019, 3:56:57 PM2/28/19
to Developers

Gabriel,
What you propose is very limited and requires modification for every individual usage case. IMO, it is not a good solution.
There are much better ways to get xxprintf() style formatted output within the C++ Arduino environment.
For example using the power of C++ and the Arduino Print class it is really trivial to make a function that works like fprintf() but uses Print class objects.
Here is the code for a simple function called Printf() that works just like fprintf() but uses a Print class object instead of a FILE pointer.


size_t Pprintf(Print &outdev, const char *format, ...)
{
char buf[128];
    va_list ap;
    va_start(ap, format);
    vsnprintf(buf, sizeof(buf), format, ap);
    va_end(ap);
    return(outdev.write(buf));
}


This function would will work with any object that uses the Print class, which includes pretty much any Serial, LCD, GLCD,  LED matrix libraries, etc..
Essentially, anything that currently supports print() will work with the Pprintf() function above.


and here is a working sketch that demonstrates how to use it with the Serial object.

void setup(void)
{
    Serial.begin(115200);
    Pprintf(Serial, "Pprintf...\n");
}
void loop(void)
{
    Pprintf(Serial, "Seconds up: %04ld\n", millis()/1000);
    delay(1000);
}

size_t Pprintf(Print &outdev, const char *format, ...)
{
char buf[128];
    va_list ap;
    va_start(ap, format);
    vsnprintf(buf, sizeof(buf), format, ap);
    va_end(ap);
    return(outdev.write(buf));
}


And here is a bit of rant on the topic of the Arduino Print class.


From a larger perspective, IMO, the only reason not to have a printf() method built into the bundled Print class provided by the Arduino IDE is obstinance on the part of certain key Arduino developers. Yeah I know that is a pretty harsh assessment, but that is my opinion.
At this point in time, the xxprintf() formatting routines have existed for 50 years.
They are very well documented and understood and examples on how the formatting string works are literally everywhere.
Many people have been asking for it on Arduino for many years.
This playground page has existing for around 6 years: https://playground.arduino.cc/main/printf
Also, several 3rd party cores have already added a printf() method to their Print class. Cores like Teensy/Teensy3, Chipkit, and ESP.
Even if very green/newbie Arduino users find it complicated or too complex for them, they don't have to use it and it won't eat up any code space in their images.
But in reality with a small bit of coaching and a bit of googling, I believe that even a newbie should be able to easily figure out how to use it.

So at this point in time, I really don't understand why there is still so much resistance to adding a printf() method to the Print class.


--- bill

Adrian Godwin

unread,
Feb 28, 2019, 5:39:44 PM2/28/19
to devel...@arduino.cc
Although that suggestion is easy to implement, I'm not very keen to take 128 bytes of stack. This can and should be avoided. But to extend your 'rant' ..


What is needed is a way to link either the integer or the fp library according to the needs of the format string. Templates do get over this fairly easily at the cost of unacceptably limiting the functionality. This might be OK in the cause of simplicity if there were a reasonable way of doing formatted output instead, but the only options are to write it yourself or sprintf. Of course if you use sprintf(buf,..);Serial.print(buf); nothing is saved but some clarity.

Format inspection used to be considered impossible for printf, but modern compilers often parse the format string and provide error reports if the arguments don't match. The use of floats is therefore known at compile time. Perhaps there's a way to use this to link an appropriate library.

I don't understand the resistance either. Printf is a common C paradigm, much easier to get used to than many other features that are required (like precedence, associativity etc. which is far more likely to be misunderstood, and the different uses of &,&&,* etc.). It's (imho) superior to the frankly rubbish C++ stream idea which is unintuitive (the written code looks nothing like the output) and shows its unsuitability by the ludicrous list of hacks that are used to gain some kind of control over the precision and width. Printf has simple rules that have close analogues amongst the various forms. You don't have to understand many before they're all easy to use.

I seem to recall there was a language that had print formats in the form of an even more specific template than printf - something like nnnnn.nn. I forget what it was. A quick scan of Python, Pascal and several FORTRANS didn't find it but did confirm that there is nothing in common use  that's better than printf. Just  messier ones.

Perhaps INTERCAL is the answer.
 

--

Bill Perry

unread,
Feb 28, 2019, 8:44:06 PM2/28/19
to devel...@arduino.cc


On 02/28/2019 04:39 PM, Adrian Godwin wrote:
> Although that suggestion is easy to implement, I'm not very keen to take 128 bytes of stack. This can and should be avoided. But to extend your 'rant' ..
>

The Pprintf() function was really done as as example of a better more generic implementation of what Gabriel was doing which was using a 128 byte buffer.


>
> What is needed is a way to link either the integer or the fp library according to the needs of the format string. Templates do get over this fairly easily at the cost of unacceptably limiting the functionality. This might be OK in the cause of
> simplicity if there were a reasonable way of doing formatted output instead, but the only options are to write it yourself or sprintf. Of course if you use sprintf(buf,..);Serial.print(buf); nothing is saved but some clarity.
>
I think the PrintEX library is wonderful solution. The only issue is that it is currently only about 95% there.
Some minor tweaks, additional documentation, and better examples would make the ideal solution as it leverages on the Print class,
works on all cores, and supports floating point, doesn't require any changes to tools or other libraries, and it only pulls in the functions and support that you need.

I spent quite a bit of time looking at automating the AVR xxprintf() library code selection about 10 years ago.
I believe it can be done; though the use of some clever weak symbols and potentially some thing wrappers.
However, it can't be done, or least I couldn't figure out how it could be done, given the the way the AVR libC library and its separate versions that include different xxprintf() support were created.

One time in the past I spent several years trying to get them to fix a few things in <util/delay.h>
I worked them on the updates and fixes but it was painful and took several years to get done.

Based on that experience, I abandoned any work trying to get automatic libC library version selection as I didn't think it could be done without getting the AVR libC boys to change the way they created their libC libraries and I also wasn't sure if it
could be automated and yet still be 100% backward compatible with the existing compiler & linker flags that many projects are already using.

These days I tend to mostly use processors with cores (like the ESPxxxx)  that have a printf() method including floating point and variable field width support in their Print class or use something simple like the Pprintf() function I showed, so it isn't
as much of an issue for me as it used to be.

--- bill


Javier Mora

unread,
Jun 1, 2019, 8:21:40 PM6/1/19
to devel...@arduino.cc
El jueves, 28 de febrero de 2019, Adrian Godwin <artg...@gmail.com> escribió:
I seem to recall there was a language that had print formats in the form of an even more specific template than printf - something like nnnnn.nn. I forget what it was. A quick scan of Python, Pascal and several FORTRANS didn't find it but did confirm that there is nothing in common use  that's better than printf. Just  messier ones.
Excel numeric formats are also similar and rather straightforward.
And I don't remember how it's done in INTERCAL but I seriously doubt it's that much readable.  Then again it can't get much worse than Fortran.

And personally I think Python's str.format() is not too bad.  Maybe combining it with some form of string interpolation so that the names of the variables to print appear in the position where they are to be replaced could make for a nice formatting style.  But it seems extremely complex.

Andrew Kroll

unread,
Jun 1, 2019, 8:28:54 PM6/1/19
to Arduino Developers
lisp pretty printer anyone?


--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

Dennis Lee Bieber

unread,
Jun 9, 2019, 9:18:39 AM6/9/19
to devel...@arduino.cc
On Thu, 28 Feb 2019 22:39:29 +0000, Adrian Godwin
<artg...@gmail.com> declaimed the
following:


>I seem to recall there was a language that had print formats in the form of
>an even more specific template than printf - something like nnnnn.nn. I
>forget what it was. A quick scan of Python, Pascal and several FORTRANS
>didn't find it but did confirm that there is nothing in common use that's
>better than printf. Just messier ones.
>

COBOL PIC declarations?


--
Wulfraed Dennis Lee Bieber AF6VN
wlf...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

Bob McHugh

unread,
Jun 11, 2019, 5:32:28 AM6/11/19
to devel...@arduino.cc
Do you mean Print Using under basic?

EX:
print using "##.##" ;10.2,5.3,66.789,.234 10.20 5.30 66.79 0.23

Bob
--
You received this message because you are subscribed to the Google Groups
"Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to developers+...@arduino.cc.
To view this discussion on the web visit
https://groups.google.com/a/arduino.cc/d/msgid/developers/oia6fe14ij6r1oq1am
iubgr8oktnsu7rg0%404ax.com.

Adrian Godwin

unread,
Jun 11, 2019, 6:53:45 AM6/11/19
to devel...@arduino.cc
Could have been : I don't think I ever used COBOL.

However, there's another huge advantage to using printf in a C-based language : testing.

Although many Arduino sketches aren't portable because of heavy dependence on hardware, the non-IO code can often be written and tested in isolation using a traditional C environment with files or static data replacing realtime data with a simulation.

It's a real nuisance swapping the debugging (algorithmic debugging imho is much better done with continuous data than an interactive debugger) between printf  and serial.print, especially given the clumsy formatting of serial.print that turns a single statement into several.

So even if a practical alternative to printf is available, it needs at the very least a library to allow code written for it to run in a stdio environment without the need for a lot of conditional compilation.
 

Bob McHugh

unread,
Jun 11, 2019, 4:03:46 PM6/11/19
to devel...@arduino.cc

On testing...

 

This isn't so much Arduino as it is all computers in general, I have a similar system on multiple platforms.  I always did conditional assembly -   /ddodebug  under multiple languages

which you can use to remove all debug based code if you place the conditional within the macro

 

The problem always arises when you are dealing with clock cycle intensive work, or within an interrupt you needed to track - your stuff needs to be FAST if working with system calls

 

I used to have one I wrote for assembly under 8088:  essentially it was a macro in the form of

 

macro Debug  <msg1>

#ifdef dodebug

  call debugprint

  db Msg1,0

#endif

endm

 

---->>> Where debugprint was linked as a separate routine in your code, and /ddodebug was the assembler conditional directive

or

 

macro Debug <msg1>

#ifdef dodebug

  Int 60  (any open interrupt you have, place your receiver routine on that interrupt)

  db Msg1,0

#endif

endm

 

---->>> Where receiver was sitting on an interrupt to be interpreted as a separate routine in your code

 

So you could write    Debug <'Location 1:  %h@pc  values: %h@ax %h@bx %h@cx %c13,10'>  (in 8088 for example)

 

Your receiving routine saves the return address and pops of the message to the return location.  With the interrupt one popping more information off the stack.

You should note that any debug that does anything major should first save all of the register values to be restored and switch to a new stack immediately to not corrupt the primary code stack in process.

 

Within that debug routine I had %c %h %o %d %s print for char, hex, octal, decimal, and string respectively, and looking up register values from the text.  That worked well

 

If you get creative you can write another debug receptor that allows you to dump the output to memory, file, serial, or a monochrome monitor for example as the output at the end of the debug section.

 

I wrote a whole subsystem for DOS that dumps all of the file system calls (selectivity) and allows you to direct them to where you want using this method.

 

It does effect model size of course under those OS's, they have compact, small, medium, and large compiler models.  With the code and data being restricted accordingly, just like C.

 

Maybe you could develop a similar system for Arduino.  Serial by the way can be a problem if you have any sort of handshaking to outside devices - it will slow it quite a bit, same with disk output,

 

You might even setup a secondary Arduino and send the output to it and let it do the debug output work, maybe to a VGA monitor or something.  You would want to setup a ring buffer on the primary so as to not lose anything, but I bet it could be done fairly easily.

 

Hope that gives you some ideas. 

 

B

 

 


Ian Katz

unread,
Jun 12, 2019, 6:52:14 AM6/12/19
to devel...@arduino.cc
Can you give an example of such a test? In other words, what behavior
(or sequence of events) are you trying to validate with the debug
statements?
> To view this discussion on the web visit https://groups.google.com/a/arduino.cc/d/msgid/developers/8776F1A6B85C4627AB6412602B5570EB%40JohnDoughPC.
Reply all
Reply to author
Forward
0 new messages