The first step is to scan the material with your favorite scanner and try to already make a good job of it. Typically, even if you did your best, it’ll end up looking something like:

Which isn’t too bad but 1) is somewhat skewed and 2) shows uninteresting things such as the book’s binding and other background elements. We want to crop that automagically and deskew the image. To deskew the image, I use Fred Weinhau’s textdeskew script, built upon Imagemagick and Awk. Unfortunately, crop isn’t quite automated yet. The script looks like:

#!/usr/bin/env bash for z in *.png do convert $z \ -crop 1625x2600+50+50 \ -gravity center \ -background white \ -extent 1800x2700 \ +repage tmp.png ./text-deskew.sh tmp.png crop-rot-$z done

This produces this image:

Now, if we’re going to bundle the images into a DjVu file, filling the border with white might not be the best course. However, if you rather intend to assemble the image in a somewhat lightweight PDF file, you might want to threshold it into a b/w image. To do so, I use Imagemagick. But Imagemagick is full of idiosyncrasies and is especially finnicky.

Here’s the thresholding script:

#!/usr/bin/env bash base=$(dirname $0) files=($(( for f in $@ do echo $f done ) | sort)) c=0 for z in ${files[@]} do # threshold convert -verbose \ "$z" \ -unsharp 5x5 \ -colorspace gray \ +dither \ -remap ${base}/bw.gif\ -normalize \ -threshold 70% \ png:bw-"${z//.*}.png" done

The script applies an unsharp mask to enhance somewhat details, then uses another image’s palette (a 1×1 pixels GIF containing a 2-color palette) to remap the colors of the original image after normalization and thresholding.

You can download the 1×1 pixels bw.gif here.

At last, we get the final image:

*

* *

These scripts are somewhat brittle, principally because Imagemagick is *really* finicky. The order of the options passed to the `convert` command has to be exactly right, otherwise it won’t work, it’ll do some other random thing. So, use at your own risks, an all that.

Filed under: Bash (Shell), hacks Tagged: Document, ImageMagick, Scanning, thresholding, Unsharp, Unsharp Mask ]]>

There are two main components to the script: 1) finding the files, which is readily done using the dreadfully capricious command `find` (which, methinks, is one of the four horseman of the scriptocalypse alongside `sed`, `awk` and `bc`) and 2) `sort` which as the oxymoronic option `--random-sort`, which, as it implies, randomizes the input lines.

In this particular occasion `find`‘s usage is minimal: a root directory under which the files are to be found and a mask, in this case, `*flac`. To deal with spaces in file-names, `find`‘s output is passed, via a pipe to `sort --random-sort`, whose output is piped to `head` (with an argument) to extract the first *n* files. This output is in turn piped to a subshell that reads, line by line, file-names and applies deflaculation.

#!/usr/bin/env bash location=$1 if [ "$location" != "" ] then nb=${2:-20} pattern=${3:-flac} find $location -iname '*'$pattern \ | sort --random-sort \ | head -n $nb \ | \ ( while read filename do bn=$(basename "$filename") nw=${bn%.*}.wav echo -\> $nw flac --silent --decode "$filename" -o "$nw" done ) else echo must provide location/directory 1>&2 exit 1 fi

Here, the only real trick is to pipe, line by line, file-names to a subshell. This is, maybe, the simplest way to deal with a filename such as “DJ Champion — N⁰1 – 013 – Grand Prix.flac”.

There’s also a Bash substitution pattern, `${bn%.*}` that strips the file’s basename of its extension.

You also might have noticed the unusual patterns in `nb=${2:-20}` and `pattern=${3:-flac}`, which provides default values if the second and third script arguments (`$2` and `$3`) aren’t provided.

Filed under: Bash (Shell), hacks Tagged: bash, DJ Champion, Find, FLAC, head, pipe, script, sort, subshell, WAV ]]>

That’s the idea behind μ-law encoding, or “logarithmic companding”. Instead of quantizing uniformly, we have large (original) values widely spaced but small (original) value, the assumption being that the signal variation is small when the amplitude is small and large when the amplitude is great. ITU standard G.711 proposes the following table:

Range | value spacing | Code |

4063 — 8158 | 256 | 1000xxxx |

2015 — 4062 | 128 | 1001xxxx |

991 — 2014 | 64 | 1010xxxx |

479 — 990 | 32 | 1011xxxx |

223 — 478 | 16 | 1100xxxx |

95 — 222 | 8 | 1101xxxx |

31 — 94 | 4 | 1110xxxx |

1 — 30 | 2 | 1111xxxx |

0 | — | 11111111 |

-1 | — | 01111111 |

-31 — -2 | 2 | 0111xxxx |

-95 — -32 | 4 | 0110xxxx |

-223 — -96 | 8 | 0101xxxx |

-479 — -224 | 16 | 0100xxxx |

-991 — -480 | 32 | 0011xxxx |

-2015 — -992 | 64 | 0010xxxx |

-4063 — -2016 | 128 | 0001xxxx |

-8159 — -4064 | 256 | 0000xxxx |

This table gives a code for about 14 bits per sample, which was deemed enough for toll-quality sound. But what if we want a code for 16 bits? The above table doesn’t quite cover the range. We could reduce the precision of the sample from 16 to 14 (with techniques like those we saw last week) then use G.711 μ-law companding. Or we could redesign the code.

Let’s say we still want a similar code structure with sign, range and value within range, that is:

s | rrr | xxxx |

Let’s also remove the special cases for 0 and -1. Let’s also call it the S-law, because of reasons. We must now adjust the “stretching”. Knowing that the lowest number in a range is 1 more than the largest number in the previous range, we find that we must solve

for . The exact value of , 2.789469…, is rather cumbersome for our needs. Such a value spacing would lead to non-integer values and we kind of want to avoid that. If we round down to two, we get much less than the desired range, and 3 gives us more. Using 3 we the following code:

Range | value spacing | Offset | Code srrrxxxx |

17488 — 52479 | 2187 | 1093 | 0111xxxx |

5824 — 17487 | 729 | 364 | 0110xxxx |

1936 — 5823 | 243 | 121 | 0101xxxx |

640 — 1935 | 81 | 40 | 0100xxxx |

208 — 639 | 27 | 13 | 0011xxxx |

64 — 207 | 9 | 4 | 0010xxxx |

16 — 63 | 3 | 1 | 0001xxxx |

0 — 15 | 1 | 0 | 0000xxxx |

-16 — -1 | 1 | 0 | 1000xxxx |

-64 — -17 | 3 | -1 | 1001xxxx |

-208 — -65 | 9 | -4 | 1010xxxx |

-640 — -209 | 27 | -13 | 1011xxxx |

-1936 — -641 | 81 | -40 | 1100xxxx |

-5824 — -1937 | 243 | -121 | 1101xxxx |

-17488 — -5825 | 729 | -364 | 1110xxxx |

-52480 — -17489 | 2187 | -1093 | 1111xxxx |

This leaves us codes 0x78 to 0x7f and 0xf8 to 0xff for other uses, as they represent values above 32767 and below 32768. The offset is needed for reconstruction: it gives the center value for that cell. A cell that is 3 units wide has its center at 1 (starting at zero, of course), a cell that is 9 units wide has it center at 4, etc.

Let’s see what the code should look like:

//////////////////////////////////////// uint8_t s_law_compress(int16_t s) { if (s<=-17489) return 0x80 | 0x70 | -(s+17489)/2187; if (s<=-5825) return 0x80 | 0x60 | -(s+5826)/729; if (s<=-1937) return 0x80 | 0x50 | -(s+1937)/243; if (s<=-641) return 0x80 | 0x40 | -(s+641)/81; if (s<=-209) return 0x80 | 0x30 | -(s+209)/27; if (s<=-65) return 0x80 | 0x20 | -(s+65)/9; if (s<=-17) return 0x80 | 0x10 | -(s+17)/3; if (s<=-1) return 0x80 | 0x00 | -(s+1); if (s<16) return 0x00 | 0x00 | s; if (s<64) return 0x00 | 0x10 | (s-16)/3; if (s<208) return 0x00 | 0x20 | (s-64)/9; if (s<640) return 0x00 | 0x30 | (s-208)/27; if (s<1936) return 0x00 | 0x40 | (s-640)/81; if (s<5824) return 0x00 | 0x50 | (s-1936)/243; if (s<17488) return 0x00 | 0x60 | (s-5824)/729; /*else*/ return 0x00 | 0x70 | (s-17488)/2187; } //////////////////////////////////////// uint16_t s_law_decompress(int8_t c) { uint8_t range=c & 0xf0; // srrr uint8_t value=c & 0x0f; // vvvv switch (range) { case 0xf0: return -17489-value*2187-1093; // 0xf8--0xff undefined! case 0xe0: return -5825 -value*729 -364; case 0xd0: return -1937 -value*243 -121; case 0xc0: return -641 -value*81 -40; case 0xb0: return -209 -value*27 -13; case 0xa0: return -65 -value*9 -4; case 0x90: return -17 -value*3 -1; case 0x80: return -1 -value -0; case 0x00: return 0 +value +0; case 0x10: return 16 +value*3 +1; case 0x20: return 64 +value*9 +4; case 0x30: return 208 +value*27 +13; case 0x40: return 640 +value*81 +40; case 0x50: return 1936 +value*243 +121; case 0x60: return 5824 +value*729 +364; case 0x70: return 17488+value*2187 +1093; // 0x78--0x7f undefined! } return 0; // unreachable, but needed to suppress warning }

*

* *

I cannot really measure how good the code is subjectively. The only thing I can say about it is that hiss is hardly noticeable, much less than a simple truncation to 8 bits. Well, that’s not that surprising: μ-law and α-law (essentially the same idea but using different constants) have been standardized and used for protocols where the target bit rate was 64 kbits/s (at 8000 samples/s, and 8 bit per sample).

Still, let’s have a look at the effect this companding has on sound. I took Bizet’s “Les toréadors” (most because I needed something I could stand listening to 20 times in a row). The original piece, fresh of the CD looks like:

As one can see, the original has been very carefully mastered to remove any low amplitude high frequencies, that is, *noise*. Random low amplitude high frequencies component would manifest itself as high pitch hiss, and can be noticeable if the music doesn’t mask it.

With a reduction from 16 to 8 bits then back again, the hiss is conspicuous:

The blue/purple/gray texture shown in the spectrogram is akin to white noise, low amplitude in all frequencies. You can hear a distinct hiss. With S-Law, we see that there is a lot of noise:

But: 1) it’s much better at reducing noise with very low amplitude sound, as is shown at the extreme left: the leading “silence” remains (mostly) silent, while with 8 bits downgrading, there’s a fair amount of noise in that area. 2) The total quantity of noise is also much reduced, as is shown by the prevalence of pale blue areas.

*

* *

Of course, 8 bit single-sample companding can’t do miracles. That’s why this type of approach is more or less forgotten and more complex coding techniques are favored. One big difference is that modern waveform-coding techniques will not code individual samples but a bunch of samples together, exploiting all kind of dependencies.

Filed under: algorithms, bit twiddling, data compression, hacks Tagged: alpha-law, companding, ISDN, mu-law, noise, sound, spectrogram, voice, white noise ]]>

So merely shifting the values isn’t sufficient. We must make sure that 0 maps to 0 but 255 maps to 65535. Luckily, we notice that 65535 is divisible evenly by 255, and that 65535/255=257. This gives the conversion pair:

int16_t _8_to_16 (uint8_t x) { return x*257; } // promotion to int uint8_t _16_to_8 (int16_t x) { return x/257; }

That’s nice, but that’s not always exactly the code we need. A peculiarity of the WAV format is that samples on 8 bits are from 0 to 255, but samples on 16 bits are from -32768 to 32767. Therefore, we need to worry about that bias. And, while we’re at it, let’s not trust the compiler to promote the operands to (a 32 bits) int:

int16_t _8_to_16 (uint8_t x) { return ((int32_t)x*257)-(1<<15); } uint8_t _16_to_8 (int16_t x) { return ((int32_t)x+(1<<15))/257; }

Spiffy. Now, what about, say, 16 to 24 and back? 65535 doesn’t stretch as well on 16777215 as 255 did on 65535: 16777215/65535=256.0038…, so it wont be as easy. However…16777215/65535 is also 65793/257:

int32_t _16_to_24(int16_t x) { int64_t t=x; return ((t+(1u<<15))*65793)/257-(1u<<23); } int16_t _24_to_16(int32_t x) { int64_t t=x; return ((t+(1u<<23))*257+256)/65793-(1u<<15); // with a bit of rounding! }

The rounding make the thing perfectly reversible: `_24_to_16(_16_to_24(x))` always gives back `x`, which is nice.

*

* *

You might object—rightfully so—that integer operations such as multiplication an division aren’t free, especially on weak processor, and even more so with weird constants like 65793. OK, let’s do this with additions and shifts then.

int16_t _8_to_16 (uint8_t x) { return ((x<<8)+x)-(1u<<15); } uint8_t _16_to_8 (int16_t x) { return (x+(1u<<15))>>8; } int32_t _16_to_24(int16_t x) { int32_t t=x; t+=(1u<<15); return ((t<<8)+(t>>8))-(1u<<23); } int16_t _24_to_16(int32_t x) { return (x>>8); }

To stretch 0 to 255 on 0 to 65535, we may notice, if we think in hex, that since 255 is 0xff and 65535 is 0xffff, 254, being 0xfe should stretch to 0xfefe (which indeed it does: 254*65535/255=65278=… 0xfefe! To get back the original 8 bit value, we need only the 8 most significant bits, so a shift right by 8 bits does the trick. In the code above, we also take into account the fact that for WAVE files, 8 bits samples are from 0 to 255 while 16 bits samples are from -32768 to 32767.

The conversion from 16 to 24 bits (and back) uses the same trick, copying the most significant bits onto the missing lower bits. Again, quite easily reversible. The code for 16 to 24 bits must compensate for sign before adding the least significant bits (that’s why merely setting the least significant bits with an or wouldn’t work). The code for 24 to 16 bits is even simpler, because the bias is also correctly shifted! Wunderbar.

Filed under: algorithms, bit twiddling, hacks Tagged: 16 bits, 24 bits, 65793, 8 bits, bits per samples, Samples, sound, WAV ]]>

IEEE 754 floats, the most likely implementation of the floats on your computer, aren’t as straightforward as one might think. First, they use a “scientific notation” representation of numbers, with a sign bit, a certain number of bits for the exponent, and some more bits for the mantissa, the “precision bits”. However, since they’re finite-precision numbers, they’re bound to exhibit all kind of error propagation when used in computations.

Even the simple question of the sum of a series, say a vector, of floats isn’t quite that simple. Indeed, we may be tempted to answer the question (how to compute the sum of a series of floats) with something like this:

float simple_sum(const std::vector<float> & v) { return std::accumulate(v.begin(),v.end(),0.0f); }

with `std::accumulate` being defined in `<algorithm>`. It does what you expect: it sets a temporary to zero, then adds one by one the items of the collection to this temporary, and returns it when it’s done. This code is simple, container and type agnostic. It should be fine, except it isn’t.

To understand what’s wrong with this simple code, we must first understand how float addition works. As I mentioned earlier, a float is expressed as

,

where is the sign bit, the exponent (stored as 8 bits) and the mantissa, that keeps only the 23 most significant bits of the number (well, 24, because the most significant bit being always 1 needn’t be stored).

When you add two numbers, if the two exponents are about the same, then the tow mantissas more or less align and the addition works out correctly. Neglecting exponents, and keeping only the 23 most significant bits:

11000010110000001100000 ... + 1111000001010000111000 ... = 11010001110001011100111 ....

This isn’t very different of what we do when we compute long sums on paper; the only difference is that the least significant bits that are missing are taken as zeroes.

However, if the exponents differ somewhat more than 23, we find ourselves in a this situation:

11000010110000001100000 ... + 1111000001010000111000 ... = 11000010110000001100000 ...

and the sum is not the sum of the two numbers, but only the greatest of the two! Now, imagine that in your collection, you have one extremely large number and the rest of the series is composed of smallish numbers. The naive sum isn’t the sum at all: it more or less computes the maximum of the series!

How can we work our way of this? The short answer is “we can’t”. The longer answer is that we must find an order to perform the sum in which only numbers of comparable magnitudes are added together. Sorting them (in increasing order) doesn’t seem like a bad start. Let’s see.

First, let’s create a troublesome array:

#include <cstddef> // size_t #include <vector> #include <random> // for some weird reason, generators and // distributions do not inherit from common // ancestors; template is pretty much the // way to do this. template <typename D,typename G> void fill(std::vector<float> & v, size_t nb, D & distribution, G & generator) { v.resize(nb); for (auto & f : v ) f=distribution(generator); } .... int main() { std::default_random_engine generator; std::exponential_distribution<float> exponential(0.125f); std::vector<float> v; fill(v,1000000,exponential,generator); ... }

This fills the `std::vector` with exponentially distributed values with , what is, with an average of , but with some very large values. This should provide a “troublesome” series.

We’ve already presented the code for the naive sum. Let’s show it again:

float simple_sum(const std::vector<float> & v) { return std::accumulate(v.begin(),v.end(),0.0f); }

This, as mentioned earlier, just do the naive sum. But we now that if the next number to add is too small, it won’t be accounted for. So let’s pursue this idea we had earlier: sort them first then add them:

float sorting_sum(const std::vector<float> & v) { std::vector<float> t(v); std::sort(t.begin(),t.end()); return simple_sum(t); }

So let’s generate 1000000 values and compare the “as-is” sum and the “sorted sum”. Let’s repeat the experiment five times to see what variation there is. We might get something like:

as-is | sorted |

7995938 | 7996406.5 |

8009019.5 | 8009517.5 |

8001316 | 8001631.5 |

8009463.5 | 8009750.5 |

7995282 | 7995591 |

As expected, the sum is close to the average times the number of samples, and, indeed, the sum is always close to 8000000. But they differ! *All* of them!

*

* *

So merely sorting isn’t enough. Hmm… What else could we do? What about adding, pair by pair, iteratively, until only one value remains? What if we do a “reduction” like this:

This should, if the values are evenly distributed, end up adding numbers of comparable magnitude and help precision. A straightforward implementation might look something like this.

float pair_sum(const std::vector<float> & v) { std::vector<float> t(v); size_t l=t.size(); do { size_t d,s; for (d=0,s=0;s+1<l;s+=2,d++) t[d]=t[s]+t[s+1]; // deal with left-overs, if any if (s<l) t[d++]=t[s]; l=d; } while (l>1); return t[0]; }

And, while we’re at it, let’s make a version that sorts (increasing order) the numbers before adding them pair by pair:

float sorted_pair_sum(const std::vector<float> & v) { std::vector<float> t(v); std::sort(t.begin(),t.end()); return pair_sum(t); }

*

* *

Let’s compare the results:

as-is | sorted | pair | sorted pair |

7995938 | 7996406.5 | 7996777.5 | 7996777 |

8009019.5 | 8009517.5 | 8009789 | 8009789 |

8001316 | 8001631.5 | 8001917 | 8001917 |

8009463.5 | 8009750.5 | 8010206 | 8010206 |

7995282 | 7995591 | 7995851 | 7995852 |

If we examine the results, we see that pair-wise sums (either with or without sorting) are larger than the naive sums. This seems to indicate that more numbers are taken into account. But is the pair-by-pair sum really better? Hard to say. To compare correctly, we would need an arbitrary-precision library, but we can think it does a better job. With sorting, it come close to an Huffman-like algorithm where only the two nearest (value-wise) floats are added together. Maybe there are much better strategies. What do you think?

*

* *

The complete code: click to deflapulate.

#include <cstddef> // size_t #include <vector> #include <algorithm> // std::sort, std::accumulate #include <iostream> #include <iomanip> #include <random> // for std::distributions... // for some weird reason, generators and // distributions do not inherit from common // ancestors; template is pretty much the // way to do this. template <typename D,typename G> void fill(std::vector<float> & v, size_t nb, D & distribution, G & generator) { v.resize(nb); for (auto & f : v ) f=distribution(generator); } float simple_sum(const std::vector<float> & v) { return std::accumulate(v.begin(),v.end(),0.0f); } float sorting_sum(const std::vector<float> & v) { std::vector<float> t(v); std::sort(t.begin(),t.end()); return simple_sum(t); } float pair_sum(const std::vector<float> & v) { std::vector<float> t(v); size_t l=t.size(); do { size_t d,s; for (d=0,s=0;s+1<l;s+=2,d++) t[d]=t[s]+t[s+1]; // deal with left-overs, if any if (s<l) t[d++]=t[s]; l=d; } while (l>1); return t[0]; } float sorted_pair_sum(const std::vector<float> & v) { std::vector<float> t(v); std::sort(t.begin(),t.end()); return pair_sum(t); } int main() { std::default_random_engine generator; //std::uniform_real_distribution<float> uniform(0,1.0f); std::exponential_distribution<float> exponential(0.125f); std::cout << std::setprecision(10); for (int i=0;i<20;i++) { std::vector<float> v; fill(v,1000000,exponential,generator); std::cout << std::setw(10) << simple_sum(v) << " " << std::setw(10) << sorting_sum(v) << " " << std::setw(10) << pair_sum(v) << " " << std::setw(10) << sorted_pair_sum(v) << std::endl ; } return 0; }

Filed under: algorithms, C, C-plus-plus, C99, hacks Tagged: float, IEEE 754, Interview, std::accumulate, std::sort ]]>

There are basically three possible cases for the matrix in the equation system

:

- The matrix is square and non singular (its determinant is different from zero). In this case, the equation system is solved by
,

because the inverse of exists.

- The matrix is “tall” (more rows than columns) and its columns are linearly independent. In this case, we will need left pseudoinverse. If is tall (and its columns linearly independent), then the matrix is square and non-singular. It can be inverted. Indeed:
,

,

,

,

,

where is the best , the one that minimizes .

- The matrix is “wide” (more columns than rows) and its rows are linearly independent. This will call for the right pseudoinverse. If is wide (and its rows linearly independent), then the matrix is square and non-singular. It can be inverted. Let’s see:
,

,

,

.

The usage is to denote the pseudoinverse of a matrix as . Here, left and right do not refer to the side of the vector on which we find the pseudo inverse, but on which side of the matrix we find it. As you know, matrix product is not commutative, that is, in general we have . When the matrix is square and non-singular, the normal inverse and the right and left pseudoinverse coincide. We have . Otherwise, depending on whether A is tall or wide, we either have or .

*

* *

So what prompted me to revisit the pseudoinverse like this? Well, I was looking into old notes about quadraphonic sound, and realized that since the “compression” matrix (the one that maps four channels onto two) is “wide”, the derivation used for the channel mixing experiment (the otter story) doesn’t quite work. In fact, it doesn’t at all in this case! I foolishly relied on Mathematica, who found the correct right pseudoinverse, and use that pseudoinverse. The derivation for the *left* pseudoinverse was correct, but I needed the *right* pseudoinverse. Nemo est perfectus.

Filed under: algorithms, Mathematics Tagged: left pseudoinverse, Pseudoinverse, quadraphonic sound, quadraphony, right pseudoinverse ]]>

In pure C, we’re pretty much done for. The C preprocessor is too weak to help us built compile-time expression out of strings (or, more exactly, `const char *`), and there’sn’t much else in the language to help us. However, things are a bit different in C++.

Since C++11, a function or a variable can be defined as `constexpr`, that is, guaranteed to have value entirely defined at compile-time. This allows the compiler to compute, at compile time, the values of expressions. (This, in turn, implies that the code “compiles away” and just leaves the result.) There are a number of requirements to make a function or an expression `constexpr`, but we can still use that to make switch/case work on strings (or `const char *`).

The switch/case statement itself will still require an integral operand, so we must transform a string into an integral value. The usual way of doing this is to use a hash function. Can we built a `constexpr` hash function?

Well, yes, obviously—otherwise I wouldn’t have bothered to write that much text. One limitation of `constexpr` functions is that they pretty much have to limit themselves to declarations, directives, and a return statement. A good way of using these limitations is to think functional programming, where recursion is the main control mechanism. To hash a string (or a `const char *`) we must scan every character at least once. This suggest a hash function such as:

uint64_t constexpr mix(char m, uint64_t s) { return ((s<<7) + ~(s>>3)) + ~m; } uint64_t constexpr hash(const char * m) { return (*m) ? mix(*m,hash(m+1)) : 0; }

…where `mix` is the real hash function. I’m sure you can come up with something better, but as long as it is good enough, it’ll work just fine. So how do we use that in a switch/case? Most simply:

void switch_test(const char * str) { switch( hash(str) ) // run-time { case hash("tatatututoto"): // compile-time std::cout << "tutu!" << std::endl; break; case hash("tatatititoto"): // compile-time std::cout << "titi!" << std::endl; break; default: std::cout << "wut?" << std::endl; break; }; } int main() { switch_test("tatatititoto"); switch_test("tatatututoto"); switch_test("abababubabub"); return 0; }

This indeed produces the desired output:

titi! tutu! wut?

*

* *

The main problem with this approach is the possibility of hash collisions. The above hash function isn’t very strong, very obviously, but can be dealt with by the compiler. If a collision occurs, it will be detected at compile-time and the compiler will issue a warning or an error about “duplicate case value”. In this case, the only correct workaround is to eliminate collisions by changing the hash function. Anyway, I’m sure you can do better than the `mix` hash function above.

Filed under: algorithms, bit twiddling, C, C-plus-plus, hacks Tagged: C++11, compile-time, constexpr, hash, hash function, Switch/case ]]>

but that expansion is only valid for , or so, because it is the Taylor expansion of "around 1", and the convergence radius of this particular expression isn't very large. Furthermore, it needs a great deal of terms before converging.

To accelerate convergence, Young and Gregory [1] propose to “compress” the series (by a short series of identities and manipulations):

,

or

.

This series, where the “around ” condition has been removed, should converge for any . However, it does not so very fast. So the problem seems to be to have a good initial guess on and use a Taylor series expansion around this good guess. Let’s revisit the first formula, valid for , but this time setting . We get

,

or

.

So, now we’re stuck with . The constant clearly depends on (since we have that ), and so can’t really be a constant. We must find a convenient from . Finding is easy when is of some special form, namely . It would be especially easy to find if (because we could use some compiler built-in, say `__builtin_clz`, to find ), but we want , not . Well, no matter, we observe that since

,

we can write

,

where , but since we choose , we get

.

Lastly, since , can be approximated as the number of bits in , something like . This operation, of course, is to be carried not in floating point (because if we’re doing series, it’s likely because we do not have a log function in the first place) but with an efficient built-in, something like `__builtin_clz`.

*

* *

Let us now compare how fast each of the variants converge.

The series “around ” with a very good (and deusexmechanically chosen) guess for converges very fast ( was chosen). The second, in green, is the series with (where the brackets stand for rounding). Lastly, Young’s and Gregory’s “compressed” series. It does converge, but it requires *a lot* of iterations.

Numerically, we also see rather rapid convergence: (with the columns in the same order: good initial guess, guess, and Young’s and Gregory’s):

*

* *

Lastly, we can ask ourselves how bad the initial guess can be given the number of iterations we are willing to perform. For now, I do not have a good characterization of it. But if we look at the following graph:

we find that, even as the number of terms grows, the convergence radius (which we should define more formally but that could be understood to be the region where the initial guess still leads to an approximation with a small error) doesn’t grow exponentially fast. More on this later, maybe.

*

* *

Choosing an initial guess using base 2 logarithms makes only sense if we have a *very* efficient way of finding integer in (where is not necessarily an integer. For this, you may use, as I said earlier, a compiler-specific extension, such as GCC’s `__builtin_clz`, that we used before, or something like it that maps to an efficient machine instruction. I also am not sure that Taylor series, and variants, are the best way of computing logarithms. However, except maybe for CORDIC algorithms, pretty much every textbook on numerical mathematics approaches the problem as we did here, except that they do not choose in “around ” adaptively.

[1] David M. Young, Robert Todd Gregory — *A Survey of Numerical Mathematics, Vol 1* — Addison-Wesley (1972)

Filed under: algorithms, bit twiddling, Mathematics Tagged: Convergence radius, Logarithm, Numerical Approximation, Taylor Series ]]>

Well, there are many wrong ways to find out. The one suggested in the tweet is hilariously inefficient, while, if it were C or C++ instead of C#, expecting definite result from undefined behavior. Yes, that `i+1` wrapping at `-1` is undefined behavior! (see, §5, ¶4, of ISO-14882:2011(E), C++11’s standard). But let’s see:

int nono() { int i=0; while ((i+1)>0) i++; return i; }

In `-O0` mode, that is, with optimizations disabled, the code takes about 4 seconds to spew out the right answer (for my computer, anyway): `2147483647`. The code generated is somewhat as expected:

0000000000400866 <_Z4nonov>: 400866: 55 push rbp 400867: 48 89 e5 mov rbp,rsp 40086a: c7 45 fc 00 00 00 00 mov DWORD PTR [rbp-0x4],0x0 400871: 8b 45 fc mov eax,DWORD PTR [rbp-0x4] 400874: 83 c0 01 add eax,0x1 400877: 85 c0 test eax,eax 400879: 7e 06 jle 400881 <_Z4nonov+0x1b> 40087b: 83 45 fc 01 add DWORD PTR [rbp-0x4],0x1 40087f: eb f0 jmp 400871 <_Z4nonov+0xb> 400881: 8b 45 fc mov eax,DWORD PTR [rbp-0x4] 400884: 5d pop rbp 400885: c3 ret

It does its ordinary stuff. Stack building preamble, sets `i` to zero in `eax`, loops, returns (implicitly) `eax` as result.

Now, with `-O3`, all optimizations:

00000000004008e0 <_Z4nonov>: 4008e0: eb fe jmp 4008e0 <_Z4nonov>

Bummer!

*

* *

So, what is the right answer?

Use the compiler and the language itself, its standard library if necessary. For the original code, which methinks is C#, `int32.MaxValue` is the right answer, always. For C or C++, we must use the limits headers. If, and only if, for some obscure reason these aren’t available, should you resort to the black arts. For example `((~0u)>>1)`, which only works if the integers are binary numbers (no, that’s not actually guaranteed by the standards for C and C++).

Type safety *is* a major issue in software development, especially when language features vary over platforms, something programming languages such as Java and C# work hard to eliminate and protect you from, but C and C++ thoroughly exploit to generate efficient code.

So, what’s the right method of getting what, exactly? Let’s refer to this table:

what | C | C++ |

Bits per char |
CHAR_BITin <limits.h> |
std::numeric_limits<char>::digitsin <limits> |

Max int(signed) value |
INT_MAXin <limits.h> |
std::numeric_limits<int>::max()in <limits> |

Size of memory block | size_tin <stddef.h> |
size_tin <cstddef> |

File offset | off_tin <sys/types.h> |
std::streamposin <iostream> |

Pointers as integers | intptr_t,uintptr_t,and ptrdiff_tin <stddef> |
intptr_t,uintptr_t,and ptrdiff_tin <cstddef> |

These are *guaranteed* by the standards to be the right types and values. Use them.

*

* *

Type safety and portability is one of the things I insist on when I teach programming. The reflex is always to use `int`, `float` or some other naïve type to store just about anything. File position? Never mind that the offsets are 128 bits, use a 32 bits `int`, then complain the code behaves weirdly when the file is over 4 gigabyte ¯\_(ツ)_/¯.

Filed under: C, C-plus-plus, C99, Portable Code, programming Tagged: CHAR_BIT, intptr_t, INT_MAX, ptrdiff_t, size_t, stddef, stdint, uintptr_t ]]>

formula for π.

Some time in the 3rd century BC, Archimedes used the perimeter of a regular polygon, starting with an hexagon and repeatedly doubling the number of sides, to estimate the value for π. He arrived at the approximation

.

How he arrived to this result is a bit mysterious until we completely understand how he got that result. Let’s see together how he did it.

First, we must understand what happens to the perimeter when we go from sides to , assuming that the polygon remains regular (that is, all its sides have the same length). Let’s see what happens, but instead of an hexagon, let’s use a square. So the first approximation is an inscribed square, whose perimeter is , for a circle of radius . If we show this in a quarter of the disk, we get something like this:

Here, . Then, if we split the side in two and push it against the circle, we get two segments of length . How do we find this new length? First, let us place the information we need on the diagram:

From this diagram, we conclude we need to find , because $latex \frac{1}{2}s is known. We have that:

,

,

,

and, at last,

,

or

.

*

* *

This suggests the recurrence

,

.

*

* *

How ever, that scary-looking BAMF^{1} simplifies quite a lot, and lets us see a lot of regularity:

,

,

,

,

,

etc.

*

* *

So we know have a formula for the length of the side in an -sided regular polygon, but that’s not the perimeter… to find the parameter, we must multiply by the number of sides:

.

*

* *

If we set , we get a perimeter ever closer to the actual circumference, and we get a better and better approximation for π. However, it’s doesn’t converge all that fast:

After 10 iterations, we have only 6 digits after the point, and at 20, 12.

*

* *

Let’s take a step back and consider how much of an achievement it was in Archimedes’ time. They did not have convenient notation, they did not have a good number system, and they had no computer. Archimedes gave up after four iterations (because he used an 96-sided polygon starting from an hexagon, he must have divided it four times, since indeed !)

^{1} For **b**ombastic **a**nd **m**agniloquent **f**ormula, of course. What did you expect?

Filed under: Mathematics Tagged: Archimedes, π, perimeter, Pi, Regular polygon ]]>