Sunday, 30 June 2019

Quantisation - Sampling the signal

I'm sure we have all heard of the concept of sampling, namely where we convert an analogue sound into digital sound. Actually, this was something that was bandied about when I was much younger, and seemed to refer to when you would take parts of songs and then meld them together to create a different song, much in the way many of the electronic musicians would do. However, in the world of data comms it is where we convert an analogue signal into a digital signal, and it usually involves sound, namely because that is the easiest.

Actually, the best way to describe how it is done is to actually show you, so let us consider the graph below (from my lecture notes, though I not will mention that from this point on, if I don't source the image, then it is either public domain, or from my lecture notes):

Actually, it looks as if it came from wikipedia: By Aquegg - Own work, CC BY-SA 3.0,
What we see above is an analogue signal with a number of blue spots on it. These spots are where we are mapping the signal. Now the whole idea of this is to digitise the signal, send it across some medium, and then when we reach the other end reproduce it as faithfully as possible.

So, the signal is being mapped as above, and at each of the points we make a record of where the signal is on the graph. As we can see, the bit level is actually 16 bits, so each of the points are translated as follows: -4, 0, 4, 7, 6, 3, 0, -1 ...

Now, I should mention sensitivity - basically that involves how close to the original signal we actually are - the greater the bit depth the more accurate the signal. I'm not sure if you actually know what a ruler is, but it is basically a length of wood (or plastic, or metal) that has two straight edges and a series of markings along the side. As well as being used to draw margins down the side of our paper in primary school, rulers were also used to measure length. Now, the sensitivity of the ruler was determined by how far apart the notches were. So, a ruler where the notches were half a centimetre apart was a lot less sensitive than a ruler where the notches were a millimetre apart.

So, once we have sampled the data, we can then convert it into 0s and 1s, but this is automatically done by our computers, the only reason we would do this is if we were writing something down on paper, or doing something a lot more detailed than simply writing a program using javascript.

However, this brings us to another problem, and that is called quantisation error, or sampling error. Basically, depending on how sensitive our readings are, we may not actually be able to sample the sound as perfectly as we will like. In fact in some cases it might actually be off somewhat. The problem is that the more sensitive the sampling is, the more bit depth is required, and the deeper the bit depth, the larger the amount of data that will be produced. As such, there needs to be a trade off - smaller files produced a lesser quality while a greater quality produces larger files.

Now that we have sampled the sounds, it is time to transmit the sound across the medium. Once again we have this trade off - for a better, and faster, transmission we need a greater bandwidth, but greater bandwidth leads to greater cost. Yet, if we lower the costs by lowering the bandwidth, we produce not so much less quality sound, but also distortion. Look, that is probably not too problematic if we aren't listening to the sounds live, since we can buffer it, but when it comes to live transmissions, then we need the higher bandwidth.

The next problem we need to consider is how often we sample the sine wave. Consider the graphs below:

Looking above we can see if we take a sample every half frequency we basically get absolutely nothing. If we take it 2/3 of the frequency, we can something but it looks like it is simply going to be a jumble of noise. Looking at the next one, I'm not sure if the 'wave frequency' is correct, because if we took it at the wave frequency, it would be the same the half frequency. What looks like is being done here, is that it is being taken at four times the frequency, and the next one at eight times the frequency. Here we are getting a much better sample, though obviously the more we sample, the better we get, but the problem of the trade off then arises in that the more we sample, the larger the amount of data that is produced.

In fact, the sampling rate is also measured in hz, being the number of samples per second. So, once again, the greater the number of samples that are taken, the better the quality. The standard sampling rate is something along the lines 44,100 hz.

Finally, we have interpolation, which is the way that the sound is turned back into an analogue form. Now, if we simply relied on the sampling levels, we would get a blocky, and rather horrid, sound. So, interpolation is an algorithm that is used to make the wave as smooth as possible to reproduce the sound in the best way possible.

Oh, before I finish off, let us go back to our sampling at the beginning where we got the numbers: -4, 0, 4, 7, 6, 3, 0, -1.

Now, the next step is to write them down as positive numbers, so we will have 4, 8, 12, 15, 14, 11, 8, 7. Now that we have done that, we convert them into binary, however because the graph went from -ve to +ve, once we have done that we need to then place another bit in front of them where 0 represents negative, and 1 represents positive.

Creative Commons License

Quantisation - Sampling the signal by David Alfred Sarkies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. This license only applies to the text and any image that is within the public domain. Any images or videos that are the subject of copyright are not covered by this license. Use of these images are for illustrative purposes only are are not intended to assert ownership. If you wish to use this work commercially please feel free to contact me

Saturday, 22 June 2019

Stenography - Data Hiding

So, what can you tell me about this picture?

Well, you could tell me that it was painted my Leonardo da Vinci, it is located in the Lovre, that it draws so many crowds on a daily basis that it is mind boggling, or that the only reason that it is actually famous is because somebody stole it back at the beginning of the 20th century. However, what if I were to tell you that there happens to be a message hidden in this painting? No, I am not talking about the Davinci Code, but rather I am speaking about a form of cryptography known as stenography, or data hiding. This is the process of hiding a message inside something so that the message cannot be seen.

The example that our lecturer gave us was that back in the days of the Ancient Greeks, when they wanted to send a message they would shave the head of a slave, tattoo the message onto the head of the slave, and then let the hair grow back again. Honestly, that doesn't sound all that practical since it takes quite a long time for hair to grow that by the time the message was be sent, the war would be over. Still, it is an ancient example of stenography.

The benefit that stenography has over cryptography is that the message is basically hidden in plain site. Cryptography has us scramble the message with a key, but the thing is that if the message is intercepted, they basically know that there is something hidden because, well, the message is scrambled. However, if they intercept a bunch of family photos, they may not actually realise that hidden in these photos are a bunch of nuclear launch codes. Another example would be a situation in a prison. Say an encoded message is passed through from one cell to another, and the warden gets the message. Well, since the message is encoded, the warden knows something is up. However, say a book is passed through with a bookmark. Each of the prisoners has a piece of card with some holes in it. If the card is placed over the page where the bookmark is located, then suddenly the message is revealed - however, unless the warden knows of the existence of the card, then all he (or she) thinks is that the prisoners are simply sharing books.

In fact we can hide messages in quite a lot of things, whether it be paintings, audio files, or even the html code of a website. Basically the object in which we are hiding the message is known as the cover object, and when the message is hidden, the object becomes what is known as a stego-object. Then we have the stego-key, such as the piece of card in the prisoner example, which is used to retrieve the message. Finally we have the embedding and the extraction function, which is an algorithm used to place the message into the cover object, and then retrieve it once again.

Actually, there is more to stenography than just hidding messages in books to communicate behind the warden's back. It can actually be used for legitimate purposes. For instance we can hide patient data inside an ECG so that when it is sent from one place to another, all the information pertaining to the patient is already there. However, simply placing the data into the ECG isn't actually going to affect the ECG all that much, if at all.

Now, as I mentioned, there is more to hiding information than placing coded messages inside the Mona Lisa. For instance you can hide numbers inside numbers. In fact you can do it in a way that it doesn't even appear that there is any message inside the number. Take this for instance, Betty wants to hide the number 3 inside the number 14256. So, we convert it into binary:

14265 = 0011100010111110;

Now, we select three random bits in the number to indicate where we are going to hide them, and in this case they will be 6,11, and 15. So, the number becomes:

0011100010111110 = 0011110010011110 = 15518.

However, let us be a little sneakier and make the number 5,10,15:

0011100010111110 = 0011100010111110 = 14256.

As you can see, by selecting specific bits we can hide one number inside another number without it actually suggesting that anything is being hidden inside the number. The key, being 6,11, and 15, or 5, 10, 15, are passed using a secure channel, or are exchanged before hand.

Actually, you don't have to stick with one number, you can actually use multiple numbers, as such:

So, as you can see, we can manipulate numbers with numbers to be able to hide numbers inside numbers. We can also hide them in a way that it is not obvious that there is anything hidden inside the number. In fact, by manipulating the least significant bit the change in the value is so small that it can go unnoticed. It is this that allows us to hide information inside things like ECG scans. To make it even more secure, only certain parts of the ECG are selected to hide the information:

Now that we see how numbers can be hidden in numbers, and also when it comes to ECGs, we can now move on to the smiling Mona Lisa and question how we can hide a message inside that smile. Well, first of all you actually need a digital image of it, namely because I'm not entirely sure if the French will be all that happy if you attempt to hide a message inside the real Mona Lisa.

On a digitial image, colours are made up of the three primary colours: red, green, and blue. Each of them will have an intensity, so say we have three values, and each of these values are represented by a four bit number, so we have a value between 000 and fff (which represents 15,15,15). 000 is black and fff is white. The first digit is red, the second digit is green, and the third digit is blue.

We then convert these numbers into binary, so we have 0000,0000,0000. Now, like the ECG above, we basically change the least significant bit of the values, and all of a sudden we have a message hidden in the colours.

Just to give you a better example, have a look at the two words below.


Both of them appear green, don't they. Well, as it turns out, one of them is off by one single bit - can you tell when one it is? No, well, that is how data is hidden inside images, or colours (though since I am writing this on blogger, actually coding the colours in the html editor is rather painful).

Hopefully that image also explains a few things.

And, so it is just a short hop to actually hiding information inside webpages, particularly when you are hiding them in the colours, as I was doing above. However, the key is to be able to do it in a robust way. While it might be easy to hide the information, sometimes extracting it can be an absolute pain. For instance, I wrote a program in javascript that placed an image onto a HTML canvas, and then proceeded to manipulate the colours to hide a message. Well, that worked okay, except when we downloaded it, and the uploaded it back to the canvas to retrieve the message, it turns out that the HTML canvas function has a complete mind of its own, and will simply set the colours as it sees fit. As such, getting the values back is no small task (a task that I am still trying to figure out).

Creative Commons License

Stenography - Data Hiding by David Alfred Sarkies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. This license only applies to the text and any image that is within the public domain. Any images or videos that are the subject of copyright are not covered by this license. Use of these images are for illustrative purposes only are are not intended to assert ownership. If you wish to use this work commercially please feel free to contact me

Tuesday, 18 June 2019

Harsh Criticism - The Misanthrope


Honestly, I'm not really all that sure whether I appreciated the Bell Shakespeare Company bringing this play into the 20th Century in much the same way that they do with quite a lot of their plays. The thing is that while the idea behind a lot of plays are timeless, somethings seeing a performance set in the original setting is so much better than having to sit through another modern Australian adaptation. Yet the truth is that it is more than possible to have this play set in the modern period, and something having such a setting does bring to light the comedy that not only made Moliere so popular, but also brought him into a lot of conflict with the powers that be (of course, the pictures on the post are not from this particular version, namely because I prefer to keep things original).

I guess the other thing we need to be aware is that this place was written in 17th Century French, so I'm not entirely sure whether performing it as it was originally written would have gone down so well with Australian audiences. As such one does need to at least translate the play, though when they introduced it by saying that it was based upon Moliere as opposed to actually being Moliere, sort of rubbed me up a little. I guess it has something to do with seeing so many of these adaptations that a part of me yearns for some originality.

However, that doesn't necessarily mean that it wasn't a good play. Sure, the back that they pretty much over used the rhyming technique really, really started to get to me. Poetry is written by meter, not by rhyme, and in a way I feel that using rhyme is somewhat childish. Hey, Dr Suess, a poetic genius who wrote some incredible children's books didn't always use rhyme (at least I believe that is the case because I don't have any of his books on hand). However, while rhyme is one way of creating poetry, it is the meter that is the one that sets the amateurs apart from the geniuses.

Yet upon reading the little booklet that I picked up at the show I discovered that the writer felt exactly the same. The thing is that the French language, as beautiful as it is, really works well when it comes to rhyme, and Moliere used it extensively. However, he was also aware that this doesn't really come across in English, and a lot of playwrights, realising that using rhyming couplets can sound pretty lame, and childish, usually dispense with it and just stick with blank verse, as Shakespeare did. However, I have to admit that it really did eventually grate with me, but that might have something to do with the Australianisation of the play.

The Critic

So the story is about that person name Alceste. In the original he is male, whereas here Alceste is female. I don't think that is all that much of an issue though. The thing is that Alceste is a huge critic of basically, well, everything, and she gets herself into a lot of trouble when she decides to voice her opinion. The play opens with her in a lot of trouble, and she is there with her best friend when she is approached by another popular artist who wants to share a song with her.

Well, out pops a country and/or western number, and of course her best friend thinks it is marvelous. Not so with Alceste as she spends here entire time ripping it to bits. Not surprisingly the artist is somewhat put off by Alceste's criticism and storms out in anger, declaring that she has not her the last of him.

As it turns out, Alceste has a boyfriend, or should I say a lover. Well, if it was in the original French setting I think lover would be an appropriate term to use, but instead we are in the Australian entertainment scene, so I'm not sure where lover, or even love interest, is all that appropriate. I'm not sure if the term boyfriend is also all that appropriate since he seems not just be a bit of a flirt, but a lot of a flirt.

Well, that isn't all that surprising, since he is one of those boy-band singers - you know, like Justin Beiber. Well, that is probably a little unfair on Beiber since he is quite monogomous, but you know the character trope. The guy that has millions of adoring teenage fans, and is more than willing to play the field behind the scenes. Yet for some reason Alceste is attracted to him. I guess it is that Alceste is human after all. Sure, she is a really harsh critic, but she also has a softer side.

So, that is basically the play. It really has a lot to do with Alceste dodging attacks from those that she has criticised, her boyfriend playing the field, and others attempting to grab his attention. Mind you, it does all fall apart for both Alceste and her boyfriend, particularly when everything comes to light. Well, despite the fact that she is one enormous crank, she also has her admirers, but as is the case, when they decide that their current pursuits are simply not worth it, her admirer is suddenly interested in somebody else.

The play finishes with the boyfriend sitting down dejected, and no longer has the heart to sing with the passion that he once did, while Alceste has basically decides to walk out on the entire scene, which two of her friends chasing after her to make sure that she doesn't do anything silly.

The Criticism

This is the thing that makes me wonder though, because criticism is part and parcel of the entertainment industry. Okay, maybe I shouldn't think too hard about it, but in the original, being set in the French court, being a critic probably did a lot more to earn enemies than it would in today's circles. Sure, nobody likes critics, particularly when they disagree with our views. In fact, I generally ignore the opinions of professional critics when it comes to movies and books because they seem to be so far away from my opinion that it is simply not worth the time to either listen to or read them. Sure, there are people that do, but I am not one of them.

Okay, Alceste pretty much hates everything, and is an absolute nitpicker, but I'm not entirely sure whether this is something that anybody could rightfully sue her for. Sure, if the attacks were personal, were lies, and where designed to ruin somebody's reputation, then sure, there is defamation, but to provide criticism for a work of art, even if the criticism is always, and without fail, bad, then I'm not entirely sure whether that will work. Okay, there are some organisations out there that go out of their way to shut down criticism - McLibel is a classic example of that - but in the entertainment industry I'm not really all that sure.

However, we can't forget that Alceste is pretty harsh in her judgements, and being a misanthrope, you can be assure that she is going to get personal at times, and this is where I suspect the problem lies. She never seems to be happy with anything, and is always looking for some flaw to point out. Actually, I'm not entirely sure whether she would be able to survive if she did, but I guess that is the issue that is coming about. The thing is that in French society one was a noble whether others liked it or not, but in the modern entertainment industry, one can be dropped in a flash, and thus your career is over - you don't even need to be a Kevin Spacey of a Harvey Weinstein for that to happen.

Though there is probably a very good reason why Alceste remained where she was. Once again it is a club, and once you have been accepted by that club, then they are unlikely to drop you, despite you being as harsh as Alceste. The thing is that everybody knows what Alceste is like, and sure, you get these newbies coming along thinking that they can challenge her, but in reality she is a sticker - she isn't going away.

That Lover

Yet there is also the issue of Cymbeline, something that grabbed my attention the first time I read the play. You see, with a play entitled The Misanthrope, I was expecting something more along the lines of Timon of Athens as opposed to a comedy. Yet Alceste is in love, and is in love with somebody who surely would be the bane of her existence. Cymbeline literally represents everything that Alceste hates about the industry, yet they are lovers (for want of a better word). Okay, Cymbeline really seems to be playing Alceste for a fool, but one has to wonder why she actually sees in him.

I guess this is the crux of the play - it does not matter how bitter and critical Alceste is, she is still a human and is still caught up with human passions. I guess that is one of those questions that will remain unanswered for ever - why do people fall in love even though the whole idea of these two people being together is absolutely absurd. For all of Alceste's criticism of the industry not only is she a part of it, she refuses to walk away from it.

Interestingly we notice how Cymbeline is destroyed at the end of the play, when his shenanigans come to light. The thing is that he really is somewhat of a philanderer, despite the fact that he assured Alceste that he was not. Alceste eventually walks away, turning her back on the whole sordid scene, while Cymbeline is left a dejected wreck, not able to sing in the way that he sung before the truth was reveal.

This is something to take note of, because the truth really has that nasty habit of coming to life. Just think of that time when you were as far away from home as you could ever be, and you run into somebody that you know. That happened to me at the Hong Kong airport, and even happened to me in Sydney, as I was greeted by a friend who was running to the airport. The thing is that we don't know who is around, who is watching, and we certainly don't know when we will encounter somebody who knows us, or something comes to light that exposes everything about us. Sure, there is all this thing about privacy, but no matter how hard we try to keep things secret, sometimes somebody is going to walk in on us when we least expect it.

The Industry

One of the interesting things about the play was that the stage opened with it completely cluttered, but ended with pretty much nothing there. It seems as if the whole facade that was the entertainment industry was being stripped away. Yet it also had a lot to do with the things that we are able to hide behind slowly being taken away until who we really are is suddenly revealed. Cymbelene is exposed as a philander, and Alceste turns her back on the industry realising that there is nothing there left for her.

Yet is the entertainment industry like the French court of the 17th Century? Well, quite possibly. The thing is that entertainers today are like the new nobility. Sure, the politicians have the member's bar, and sure, we may dislike them and distrust them, but compared to what goes on in the entertainment industry, they seem comparatively tame. The thing is that entertainers can get away with quite a lot more than people in the more respectable positions. Wild parties and ostentatious living seems to be the norm where they are concerned.

Interestingly, many of the riches to rags stories out there seem to concern entertainers more than businessmen. It seems as if businessmen are able to restrain their spending habits, and not live as outlandishly as the entertainers do. I guess that has a lot to do with entertainers being in the lime light. They have a image that they want to uphold, and they are very visible people. As such, they tend to live a lot more outlandishly in attempting to maintain that image, where as the businessmen really seem to exist and work behind the scenes.

Creative Commons License

Harsh Criticism - The Misanthrope by David Alfred Sarkies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. This license only applies to the text and any image that is within the public domain. Any images or videos that are the subject of copyright are not covered by this license. Use of these images are for illustrative purposes only are are not intended to assert ownership. If you wish to use this work commercially please feel free to contact me

Sunday, 9 June 2019

Analog & Digital Graphing

Well, before we start looking at how we can convert sounds into the electrical pulses that are required to send it along the cables that make up the network, let us consider a few more of the basic things that make up the network. Firstly we will start off with the actual cables, of which there are three types: twisted pair, coaxial, and fibre optic.

Twisted Pair: A twisted pair cable is basically what the name sounds like - two insulated copper wires twisted about each other. The advantage is that they are cheap, but the disadvantage is that not only are they slow, but they really can't send all that much along them - sort of like a really narrow pipe for pumping water. Also, you really don't want to have them running for any really long distances, or even a medium distance for that matter.

Coaxial: This is another form of copper wiring, and is what is generally referred to when people talk about the copper network, particularly when we are talking about the NBN. Look, they are perfectly fine for sending analogue voice signals, but when it comes to high speed broadband, well, they sort of aren't suited for that purpose at all. Basically it is an inner conducting wire, surrounded by insulating material, which in turn is surrounded by a metallic shield, and finally surrounded by another insulating material. ,

By Tkgd2007 - Own work, CC BY 3.0

Fibre Optic: Well, these are the cables that are considered to be at the forefront of data transmission technology. Okay, fibre optic cables have been around for a very long time, but the thing is that they are able to carry huge amounts of traffic at a very high speed. Basically they consist of cables made of glass that are insulated. In fact they will contain multiple cables bundled together. They are also very useful for sending signals long distances, and do so by bouncing the photonic signals along the cable (fibre optic uses light as opposed to electricity as a medium).

The above is what is known as guided media, namely because the signals are guided along a wire. However, we also have unguided media, which is basically radio signals sent across the, well, air. They will normally be sent from a transmitter and picked up by a receiver. The signals will be converted to electro-magnetic signals at the transmitter, and the receiver will pick it up and convert it back.

So, now let us consider the difference between digital and analogue. Basically analogue data takes of a continuous value that changes smoothly over time. Analog signals are measured in hertz (hz), otherwise known as cycles per second. Digital data takes on discrete values, usually in the form of 0s and 1s (and all the other variations that I keep on throwing up). Digital transmission is measured in bits per second. The table below shows an analogue signal on the left and a digital signal on the right.

Analog Graphs

Now, back to sine waves for a bit - they can be measured in either the time domain, or in the frequency domain. As pointed out, frequency is basically cycles per second, so a sine wave, when measured on the frequency domain, will appear different to one that appears on the time domain, as shown below:

Now, the reason we have come back to this is not so much to look at the wave on the time domain, but rather on the frequency domain. The reason for this is because it takes us into the realm of spectrum. The spectrum is the range of frequencies that the signal occupies, and if a number of signals are placed onto a chart along the time domain, we simply come up with a jumble of squiggles. However, if we place it on the frequency domain, we will be able to read it much better. The frequency domain is actually much more useful, and clearer, when it comes to reading multiple sine waves, as the diagram from my lecture notes below shows:

This is particularly useful when the three sine waves have been combined because they will actually produce something different. However, this brings us to the concept of bandwidth, which is basically the width of the spectrum. To calculate a signal's bandwidth, you subtract the highest frequency from the lowest frequency, as the diagram (from my lecture notes) below demonstrates:

So, if we have a sine wave (or actually three sine waves melded together) with frequencies of 100 hz, 300 hz, and 500 hz, and maximum amplitudes of 10 v, 3 v, and 1 v, then the bandwidth would be 500 - 100 = 400 hz, and the frequency graph would be as follows:

Digital Graphs

Well, now that we have considered the analog graphs, let us now turn to digital graphs. First, let us look at the graph:

Now, the bit interval is actually the inverse of the bit rate. So, if we have a bit rate of 2000 bps (bits per second), then the bit rate is 1/2000s = 0.0005 s = 500 μs (micro seconds).

So, now that we have the graphs, next we will move on to sampling.

Creative Commons License

Analog & Digital Graphing by David Alfred Sarkies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. This license only applies to the text and any image that is within the public domain. Any images or videos that are the subject of copyright are not covered by this license. Use of these images are for illustrative purposes only are are not intended to assert ownership. If you wish to use this work commercially please feel free to contact me

Saturday, 1 June 2019

All About Blockchain

I remember back in the mid-nineties there was this huge fad sweeping the world. People could see that it was going to be the future, and entrepreneurs were making buckets loads of money off of ideas that in reality had no chance whatsoever of actually making any money. Then, in 1998 the entire house of cards came crashing down. No, I'm not talking about the Asian tiger but rather the growing pains of the internet. Mind you, this was back in the days of dial-up modems and while security algorithms existed there were nowhere on the scale that they are today.

Well, it seems as if the bust was centuries ago, and we now live in an age where we are wandering around with computers in our pockets that are more powerful than those that sent the original space shuttle into space (and even then the space shuttle is starting to become a distant memory, as is the fall of the Berlin Wall). So, the reason that I am talking about the past is because it seems that we are seeing another technology that has the potential to change the world in which we live, and that is digital currency such as bitcoin.

In fact we have even seen what some could consider to be a bit coin crash, if this chart is anything to go by:

I guess anybody who had bought into bitcoin at the beginning of 2018 probably hasn't had all that much of a return on their investment, though the thing is that fads like this happen more often than not, when everybody pours their money into the next big thing, only to discover that the next big thing is nowhere near as great as they anticipated - just ask the Dutch about the tulips.

Anyway, there is more, in fact a lot more, to the concept of blockchain than simply cryptocurrency, though cryptocurrency is basically the thing that pretty much everybody and anybody has heard about, and think about, whenever block chain is mentioned. So, while I might touch upon crypto-currencies, here I'll be looking more at the concept behind it as opposed to the product itself.

So, What is a Blockchain

So, a block chain is a distributed ledger that is shared across a network. Each block chain carries a list of transactions, and a hash that represents all of the previous blocks in the chain. The exception to this is the first block in the chain, known as the genesis block. The way my lecturer explained it is that it is like a spreadsheet, of which there are copies on every single computer in a network. You can add rows to this spreadsheet, but you cannot change any of the existing rows. The reason that blockchain has worked so well with cryptocurrency is that it solves the double spending problem, that is the idea where you can use one bitcoin to purchase something, and then use that exact same bitcoin to then go around and purchase something else.

So, because the blockchains are replicated across a network, it makes them very difficult to corrupt. As such, this means that you can use them in what is known as a 'trustless network'. Basically they can be used even though you do not necessarily trust the source. In being able to do that you no longer need a central authority, such as a central bank in the case of currency, to regulate the chain. In a way the blockchain regulates itself by being able to authenticate itself.

Blockchain in Action

Okay, we already know about cryptocurrency, but they can be a little confusing, and rather complex, so let us look at another use for block chain - logistics. Say we have a company that produces medicine, and that the medicine needs to be kept at, or below, a certain temperature from production to its final destination at the hospital. Previously we would have had to simply trust the paperwork, but block chain actually solves that problem for us.

Basically, at every point in the production line a block is produced, and the temperature is recorded at that point. When the next stage of production commences, a new block is generated, and all of the other blocks are added to this new block. So, we have production, packaging, transportation, and finally retail. Now, just remember that all of this information is being recorded by a computer, so you don't have any dodgy truck drivers, or warehouse managers fiddling the books, so that when the medicine finally arrives at the hospital, the doctors can make sure that the medicine is okay by simply looking at the blocks. If the temperature exceeded a certain height in the process, this will be known.

So, what if the truck driver decided to go back and change the block after it was generated. Well, remember what we said about the hashes in a previous post? Well, these hashes are added to the blocks, so if anybody changes a previous block then this will be noticed since the block that had been changed will not be consistent with the block that had been automatically generated.

In fact, the temperature is just one aspect of this process because you can then also throw in a time frame as well. The time at which the medicine arrives at a certain point in the stage is also recorded, so that if they took too long to put it through one of the stages, this will also be noted in the final block, and the medicine can also be rejected. 

You see, with the traditional method, simply by going back down the chain and asking people you will basically get plausible deniability. The reality is that people, over time, learn that the best way to retain their job is to basically deny everything - in fact that is one of the key aspects of our legal system - denial. What blockchain does is that it allows us to trace where the problem arose, and to then deal with the problem at that point as opposed to attempting to dig up the truth from a bunch of people who are basically being very uncooperative.

You might also note that the hash also contains a private Id. That means that once again, only the section in which the block has been generated can actually create, or change, the block. Without the private Id, access to that particular block, or that particular part of the system, will not be able to happen.

So, here is the block chain that has basically been produced through our hypothetical logistics chain:

Now, what happens is that the temperature during the process went over the maximum value. Well, the auditor, who has full access to all of the blocks, and the private keys to the blocks, can then review each of them to see where the problem occurred. The auditor also knows all of the instructions, and as such is the only one that is able to reproduce the hashes. So, because the auditor can reproduce the hashes, the auditor can not only see that something is wrong, but also where it went wrong. For instance, consider below:

So, from looking at the blocks, the auditor can see that something went wrong with the shipment, and can then approach them and request that they produce their records. However, can the shipment change their records? Well, no, because not only don't they have access to the private key, but further, they do not have access to the retailer's system. In fact, by changing their block will produce an indiscrepancy in not only their block, but the retailer's block as well, which will once again be picked up by the auditor.

In fact there are many uses for block chain, including land registry and health care. The thing is that they all run on the same principle in that all of the previous details are encoded onto the latest details. Further, since they are stored on a distributed system, then one block can be compared with all of the other blocks in a way to prevent fraud. If you change one of the recorded blocks, this will be noticed when you compare it with all of the other blocks that have been stored on the network.

Now, bitcoin works similarly, but there is something else that needs to be considered, if we take into account the diagram below:

So, what is that Merkle Root? Well, I'm glad you asked.

Merkle Trees

Isn't it interesting that a lot of the stuff that I learnt last semester seems to be rearing its ugly head this semester, particularly the stuff from Discrete Maths. The reason I raise this is because of the concept known as the tree. Basically trees are non-cyclical graphs. Now, I'm not going to go into huge details here, but I can refer you elsewhere, such as Wikipedia (that is what is a tree in graph theory). All we need to know here is that a Merkle Tree is basically one of those trees.

So, a Merkle Tree is a hash based data structure where every leaf node is a hash, and every non-leaf node is a hash of its children. In particularly it is the generalisation of a hash tree, and in most cases is implemented as a binary tree. Take the diagram below for instance:

So, each of the leaf nodes is hashed (say that each of the leaves as a transaction that is to be recorded). Then each pair of leaves is then hashed together, and these blocks are then paired off and are hashed together, and this continues until you read the top hash, which contains the hash of all of the nodes below it. This node is known as being the hashed lists of all of the transactions that have taken place since the last block. This is also known as the root hash.

Merkle Trees are used in distributed systems where data verification is paramount. By using hashes instead of full files means that there is a saving when it comes to space. However, because they exist on a distributed system, if a block is changed in one place, then it must be changed everywhere for the change to be valid.

Now, there are two great features with regards to the Merkle Tree. The first is obvious, if we have been following along with regards to the hashes. Namely, if one of the blocks is changed then that change will be reflected all the way up to the root hash. The other aspect is not so obvious - basically if you wish to confirm that something is in the Merkle Tree, you do not need to download the entire tree - you only need to download a part of it. Namely, you only need the block header, and then the path you need to get to the transaction.

So, considering the above, if we want to validate leaf c, all we need is node H(d) and node H(ab). From leaf c, we can recreate node H(c), and then with node H(d) we can recreate node H(cd). Since we also have node H(ab) we can then recreate the root node H(abcd). As such, instead of downloading the entire tree, we only need to download two nodes, and of course the root node for comparison. This is known as 'proof of existence'.

Another example of where a Merkle Tree is used is in a system known as the IPFS, or interplanetary file system. Basically the IPFS is a distributed file sharing system that provides a distributed hash table as a form of catalogue for the files on the system. In a way it is similar to the world wide web, except that is is more like a bit torrent swarm. In fact, the distributed nature of the network means that it is much more resistant to attacks such as DDoS. When a file is added to the network it is referenced through the use of a hash, and it is through this hash that the file can be accessed.

Proof of Work

Now, it is time to start looking at what is known as a consensus mechanism, and in bitcoin this is known as proof of work. Basically a consensus mechanism makes sure that all of the members of a distributed network are the same, and provides a back up if some of the members are beginning to fail. Proof of work is basically a system where miners, through trial and error, seek to reach a consensus to be able to prove that a transaction is valid. Due to the computational power that is required to perform the proof of work, miners are rewarded with bitcoin when the transaction has been verified.

Basically, proof of work involves searching for a nounce (number only once) value that when added to a previous hash will produce a hash that begins with a specific number of 0 bits:

Confused, well, let's go through an example. Say we have a value 1034 and we want to find a hash with 4 zeros. So, we add the number one to our number, producing 10341. We then hash it producing:


This is not the hash we want, so we add a 2 instead, producing 10342:


Well, that isn't what we wanted, so let us try a 3:


It seems like I could be going on for a very long time if I continue to do it one by one, so let us jump a head and add 599 to produce 1034599:


There we go, after 599 hashes we finally have the hash that we were looking for. So, the nounce that we were looking for is 599. This is how it works with bitcoin, and the requirements keep on changing to keep the validation under the 10 minute time frame. It is the search for the nounce that is computationally expensive.

Just for another example, here is a list of hashes, from the sha256, that begin with three zeroes:

Well, this was a pretty long post, but it is a pretty detailed topic of which we have only scratched the surface. The thing is, that as with all new technologies, people seem to want to use it anywhere and everywhere. The reality is though that while blockchain may have a lot of useful characteristics, there may be a lot of situations where it may not be useful at all.
Creative Commons License

All About Blockchain by David Alfred Sarkies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. This license only applies to the text and any image that is within the public domain. Any images or videos that are the subject of copyright are not covered by this license. Use of these images are for illustrative purposes only are are not intended to assert ownership. If you wish to use this work commercially please feel free to contact me