Can self-learning AI chatbots be dangerous?

Can self-learning AI chatbots be dangerous?

Recently a report from the Facebook Artificial Intelligence Research lab (FAIR) raised quite a lot of eyebrows. Apparently artificially intelligent (AI) ‘chatbots’ using machine learning algorithms were allowed to communicate with each other in an attempt to converse better with human beings. The social media company’s experiment started off well enough, but then the researchers discovered that the bots were beginning to deviate from the scripted norms. At first they thought that a programming error occurred, but on closer inspection they discovered that the bots were developing a language of their own.

In 2016, an AI Twitter chatbot developed by Microsoft called “Tay” caused quite a lot of  embarrassment for the company when Internet trolls started to teach the bot to respond with racial messages on user questions. The aim of this experiment was also to develop an artificially intelligent entity that would be able to communicate with human beings, learn from the experience and get smarter in the process.

Press ‘4’ to wait for the next available operator…

The potential market for these AI chatbots are huge. Image if you could call your insurance or medical aid company and immediately speak to one of these bots without waiting hours for a human operator. Or navigating through endless recorded messages prompting you to press ‘1’ or ‘2’ to proceed to the next.

Imagine if these bots are able to speak to you in your own language, authenticate your identity with voice recognition and immediately understand the problem that you have. Imagine if these bots could communicate instantly with other bots on the other side of the globe to solve your problem.

This scenario is already becoming a reality, and eventually you would not even know that you are talking to a non-human AI.

Maybe Microsoft was a bit pre-mature in releasing their chatbot technology into the Internet Wild Wild West, but then again, great lessons were learned in the process. In Microsoft’s defense, they did not program the bot to be racists, nor did the bot itself have any concept of what racisms means.

Human communication

Any human language (written on spoken) might not be the most efficient way for AI entities to communicate with each other. Take English, for example. There are many words that basically mean the same thing (think vehicle/motor, or petrol/gasoline).

An AI that have to convert these words to bits and bytes to transmit over broadband Internet connections might come to the conclusion that the ones with the least amount of characters are more optimized. So it could tend to favour certain words and/or phrases.

The way that we change words and sentences to indicate tenses might also be strange to an AI. If the sentence “The boy kicks the ball” must be converted to past or future tense, an AI might device a strategy of using the character < for past tense and > for future tense. If this sentence is optimized even further, the AI could simply transmit “Boy kick ball <” or “Boy kick ball >” to indicate that the action happened in the past or the future.

So, this was precisely what the Fabebook bots were beginning to do. Below is a short sample of the new ‘language’ that they developed:

Bob: i can i i everything else . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i everything else . . . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i i can i i i everything else . . . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i . . . . . . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i i i i i everything else . . . . . . . . . . . . . .
Alice: balls have 0 to me to me to me to me to me to me to me to me to
Bob: you i i i everything else . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to

I’ve told you so!

When the general public learned about the Facebook incident, the first response was to call this a Skynet event (as per the popular Terminator movie-franchise). Indeed, these potential doomsday scenarios where artificial intelligent entities become self-aware and enslave the human race has been a popular theme of many books and movies over the years (2001: A Space Odyssey, The Matrix, I Robot etc.)

But should we be worried?

Isaac Asimov’s Three Laws of Robotics are usually quoted at this point to ensure people that there is nothing to be afraid of. However, when Asimov developed these laws, he was thinking about human-like robots or androids that would share our living space and do all our chores and dirty work. (The 3 laws are quoted at the bottom of this post).

But today the concept of robots and artificial intelligence has changed dramatically. AI entities might exist purely in a digital state without any physical form. These entities might also be decentralized, being distributed across many data centres or compute nodes – making it impossible to destroy.

The concept of ‘doing harm’ to a human being is also very vague. With social media playing such a big part in most people’s lives, cyber-bullying is just as dangerous as physical harm. Most people doesn’t bother to check the source of news events or posts and are happy to simply forward it to their followers. A malicious AI bot could easily destroy a person’s reputation by associating him/her with racists, harmful or pornographic posts and websites.

Many people have lost their jobs already by something they have tweeted.

Conclusion

Companies like your Googles, Microsofts, IBMs and Amazons (which have the funds to invest in machine learning, neural networks and other artificial intelligence technologies) are ultimately doing it to make and/or save money. I am not saying that they are not thinking about the future consequence of the software that they are developing. (The fact that the deviations of the Facebook and Microsoft bots could be identified and stopped, shows that we are still in control).

My concern is more that there is no common strategy between the different role-players with everyone essentially doing their own thing. And then there are many rogue nations and companies in the world that do not follow the rules in any case.

Chatbots and artificial intelligence are not going away anytime soon. AI will have a huge impact on our lives in the future – for the good. Lives will be saved, sicknesses healed and processes simplified because machines across the world are constantly analysing problems, learning from it and coming up with clever new solutions. But we always need to be wary of the fact that we could be creating systems that might have unexpected results as intended.

 

Asimov’s Three Laws of Robotics are as follows:
• A robot may not injure a human being or, through inaction, allow a human being to come to harm
• A robot must obey the orders given it by human beings except where such orders would conflict with the First Law
• A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws

Dealing with workplace Automation

Dealing with workplace Automation

Technology is great. What is not exciting about automated vehicles, Amazon packages delivered at your doorstep by drones or supermarkets that do not require a human cashier that have to scan each item individually?

Not a day goes by where there is not a new story about a start-up revolutionizing a process or even a complete industry; new advances in machine-learning and cloud computing or factories run entirely by robots. The unfortunate part is that for each of these success stories, real-life humans beings are losing jobs…

Rapid Population Growth

In 2016 the world’s population exceeded the 7.3 billion mark, meaning that the number of people on earth has doubled in the last 45 years.

The United Nations predicts that the figures will be somewhere in the region on 9.7 billion by the year 2050. At the current rate it would require the equivalent of almost three planet earths to sustain the population if we were to maintain our current lifestyles.

In addition to this alarming population growth, people continue to migrate to urban areas. The United Nations further predicts that by 2050, 66% of the world’s population will be living in cities. In the year 2000, the number was only 47%.

In order to sustain these numbers and the extreme demand on resources, we simply have to do more with less. In practical terms it means that we will continue to turn to science and technology to find better and faster ways to keep up with the demand. And the more we improve, innovate and automate, the less we require human beings to be part of the workforce.

These jobs are lost forever

During the last 20 years, globalization has seen many manufacturing jobs moved from First World countries to relatively poor countries that still had some basic education systems. This has been great in improving the economies of these poorer countries, but the tide is slowly turning for them as well. If one worker (with the help of a robot army) can achieve the same output as 100 factory workers, you simply don’t need to employ these 100 workers.

More production, less employees…

When Apple sells an iPhone, what percentage of the cost of the sale actually go to the labour force that produced it?

Less than 2 percent.

So when politicians throw around popular slogans like “bring back the manufacturing jobs” they are basically talking about the 2 percent labour required to manufacture an iPhone. Or a flat screen television. Or a vehicle.

Breeding ground for revolution

The net result is that millions of people worldwide are seeing local job opportunities diminish, especially in rural areas. With the migration of people to cities and jobs to foreign countries, so too has the hope of a prosperous future vanished.

Many towns and small villages’ survival depend entirely on the surrounding resources and industries – sometimes a single factory. As these industries close down, entire communities are left without income or the means of learning the skills required to work in newer “smarter” factories.

This is unfortunately a breeding ground for revolution and politicians have grasp the negative sentiments towards globalization and automation with open arms. And since a lot of the traditional voting power still lies in these effected rural areas, many politicians have found huge support in promoting anti-globalism, anti-capitalism and closed borders and markets.

Basic Income

So this is the current situation. The real question we need to ask is what we can do about it?

Fortunately for us this question is also on the mind of many wealthy and powerful business leaders. Elon Musk, the billionaire and co-founder of companies like Tesla and SpaceX believes that the answer lies in a universal basic income. This might come to a shock to many capitalists, but it means that each person on the planet receives a certain amount of money each year so that we can keep the economy going and ensure that workers displaced by automation can at least maintain a basic living standard.

But where will the money come from to fund this scheme? Bill Gates, founder of Microsoft and present-day philanthropist propose that we should tax the same companies that are displacing the workers in the first place. Again, this might not please the owners of such companies but is it really fair that only a select few benefit from a system while millions other suffer?

The problem with all these people losing jobs is that it has a rippling effect through the economy. Middle-income workers might belong to a medical aid, go to restaurants once or twice a month or can afford to put their children in private schools. Without an income, these types of luxuries will be the first to be given up in order to make ends meet.

But just think for a moment how many other jobs will eventually be lost in the process – administration clerks working for that medical aid, the groundsmen of the private school, and the kitchen staff of the struggling restaurant. The cuts will come with the “low hanging fruits” first – jobs that could in itself be replaced by robots or simply be dropped in order to pursue profit.

Even low-income workers that support local economies and spaza shops will change their buying and lifestyle habits. And as all these workers lose the ability to contribute to the economy or fall out of tax-paying brackets, where will governments get the money to support their population?

Robot workers don’t pay taxes, don’t belong to medical aids and don’t buy Christmas gifts from local arts and crafts markets. So I think Bill Gates is definitely on to something.

Utopian Society

How will a world look like where robots are doing all the work while humans get paid?

It sure sounds a lot like the plot of some utopian science fiction novel. I personally think that society as a whole will uplift themselves to a higher level because without the daily struggle to provide basic needs, people can pursue other passions. Entrepreneurs and business-minded people will still seek opportunities to make money or provide services but now people that previously had no way of entering the economy suddenly can be part of it. Bill Gates also believes that we could focus more on humanitarian work, looking after our children and their education as well as caring better for the elderly.

Yes, there will always be crime, corruption and other social problems – as there has been throughout history. But there will also be people that seek higher forms of fulfilment like art, music, literary and poetry.

We hold the cards

I have read somewhere that one of the last jobs that will be replaced by robots, will be that of the politician! The irony is the same people that has the power to do something about workplace automation will make sure that their own jobs are protected until the bitter end.

So why not make use of that power for a greater good and start having a serious look at ways to address the issue? We know that the problem will only get worse in the future, so now is the ideal time to come up with a solution.


 

Other sources:

The World Population Prospects: 2015 Revision (http://www.un.org/en/development/desa/publications/world-population-prospects-2015-revision.html)

http://www.un.org/en/development/desa/news/population/world-urbanization-prospects-2014.html

“Those jobs are gone forever. Let’s gear up for what’s next.” (Quincy Larson, Medium, 2017-02-06).

 

 

 

 

Space Travel and Science Fiction

Space Travel and Science Fiction

I have always been a huge fan of science fiction. What really draws me to the genre is the fact that authors and movie-makers are not scared to “challenge our minds”, exploring ideas and concepts that might seem implausible, even ridiculous today. As Robert A. Heinlein puts it: “(Science fiction) is a realistic speculation about possible future events, based solidly on adequate knowledge of the real world.

A popular saying is that “today’s science-fiction is tomorrow’s science fact”. The irony is that there is more truth to this statement than we would care admit. 30 years ago the ‘portable telephone’ and ‘information-highway’ were something of science-fiction. Today there are more cell-phones on the planet than people with the Internet being the number one source of information, communication and entertainment.

For this post, I am investigating some of the science fiction predictions for space travel – from the absurd to the “somewhat-plausible” to the “it-is-only-a-matter-of-time”.

Leaving Terra Firma

To get into space, you first need to overcome the effects of gravity – one of the fundamental forces of the universe that effects all matter at a macroscopic level.

Currently, the only technology available is rockets which is tremendously expensive – so much so that only governments, multi-national consortiums and the richest billionaires on earth are able to fund space projects. Rockets need to generate enough thrust to escape the pull of the earth’s gravity (g = 9.80665 m/s) and have to burn enormous amounts of expensive fuel in the process.

In his 2006 novel, Gradisil, Adam Roberts explores the idea of ordinary people being able to simply “fly” into orbit, setup habitats and live in space – which they call the “Uplands”. In order to do so, regular jet-planes are adapted to use the resistance of the earth’s magnetic field to produce lift. He called this effect, ‘magnetohydrodynamics’ where the plane’s wings becomes giant electro-magnets that cuts into the lines of force of the magnetosphere. The ascent is slow and similar to a seagull or eagle using the wind currents to systematically soar higher and higher.

Whether or not this technology is viable for future space travel is debatable. However, it provides an interesting alternative to current space programs’ investment in rocket technology. As one of the characters in the book explains (rather animatedly): “…because von Brown (the ex-Nazi scientist that eventually spear-headed the NASA space program – editor) was so influential, nobody explored other means of flying to space. When NASA planned to fly a hundred-kilo man into the orbit they could have taken the money they were going to spend on doing that and instead spent it on building a replica of the man in solid gold, that’s what the costs were.”

Space Elevators

Another alternative for getting into orbit is the space elevator. The concept of space elevators were first proposed in 1895 by Konstantin Tsiolkovsky. Arthur C. Clarke introduced it to the wider science fiction audience in his 1979 novel, The Fountains of Paradise.  Robert A. Heinlein (Friday, 1982) and David Gerrold (Jumping Off The Planet, 2000) also featured space elevators that reached into the sky like giant beanstalks.

The dynamics of such an elevator is not hard to grasp. A counter-weight needs to be placed into the earth’s orbit and connected to an anchor point on the surface with a strong cable or tether. The centrifugal force of the earth’s rotation will keep the counter-weight in place (think a bucket of water that you swing over your head without spilling any fluid).

Tsiolkovsky’s vision of the elevator was a bit more ‘anchored’ in his era’s understanding of physics and construction and consisted of a free-standing tower that reached the height of the geostationary orbit (like a giant Eiffel Tower). But today’s thinking is more towards using a very strong but light-weight cable that is capable of carrying its own weight as well as the additional payload that must be lifted. It needs to be about 35km long. The material has not been perfected yet, but scientist are currently working on carbon nanotube and diamond nanothread technologies.

The benefit of such a space elevator is that all the raw materials needed to construct spaceships can be exported from earth where it will be assembled in zero-gravity (by means of robots or humans).

In order to make it economically viable, such an elevator would probably consists of hundreds of cargo pods that move up and down together like a giant Ferris-wheel. It would obviously mean that a journey to the top could take many days as each pod needs to be unloaded at the top while an empty one is loaded at the bottom. But the money saved in relation to burning rocket fuel would be worth the while.

Space Launch

So let’s assume a new spaceship has been assembled outside the earth’s atmosphere or even on the moon. The next step is to launch it towards its destination – an action that also requires some sort of energy transfer.

In Kim Stanley Robinson’s 2015 novel Aurora, a star ship with the same name is launched by an electromagnetic “scissors” field. Two strong magnetic fields held the ship between them and when they were brought across each other, the ship was briefly projected at an accelerative force equal to 10 g’s. (Almost like squeezing a watermelon seed between the fingertips).

In addition to this, a powerful laser beam were concentrated on a capture plate at the stern of the ship’s spine for a period of 60 years, accelerating it to full speed.

But there are other objects in space like planets, moons and even suns whose energy could also be harnessed to propel spacecraft on long interstellar travels. The most famous example is Voyager I.

Voyager I was launched in 1977 and is the first human-made object to cross the heliopause (the boundary of the heliosphere) and enter interstellar space. On his journey to the edge of our solar system, Voyager had a little help from the gravitational fields of Jupiter and Saturn to sling-shot it towards its destination.

The mathematics behind such a manoeuvre is extremely difficult and commonly known as the “three body problem” – a reference to how the gravity of the sun and a planet will influence a third object’s trajectory. A brilliant 25-year-old mathematics graduate called Michael Minovitch solved this problem in 1961 (with the help of an IBM computer) and proved that an object that flew close to a planet could steal some of the planet’s orbital speed, and be accelerated away from the Sun.

Another problem is that the planets are not always in the correct position to execute the sling-shot. The Voyager missions were specifically planned to take advantage of the fact that Jupiter, Saturn, Uranus and Neptune would all be on the same side of the Solar System in the late 1970s. Such an opportunity would not present itself again in another 176 years!

In Aurora, Kim Stanley Robinson also used the inverse of the sling-shot effect to decrease the speed of his spaceship as it re-entered the solar system by using the planets’ gravitational fields for ‘aero-braking’. His ship also had a very intelligent computer on board that was able to make the required calculations during flight.

Bistromathic Drive

In Douglas Adam’s Hitchhiker’s Guide to the Galaxy series, he introduced us to the Bistromathic Drive which operates on a revolutionary way of understanding the behaviour of numbers. Just as Einstein observed that space and time was not absolute but depends on the observer’s movement in time, scientists discovered that numbers were not absolute, but depended on the observer’s movement in a restaurant.

The first non-absolute number is the number of people for whom the table is reserved. This will vary between the reservation, the actual amount of people showing up, the number of people joining the table during the evening and the number of people leaving the table when they see someone else turning up which they do not like.

The other non-absolute number is the arrival time – a number whose existence can only be defined as being anything other than itself. In other words the exact time when it is impossible that any other member of the party will arrive!

The novel further explains how these and other numbers are utilized in the drive so that the ship is capable of travelling two thirds across the galaxy in a matter of seconds.

Ion drives and more

Although the bistromatic drive will not be in production any time soon (or at all!) ion drives (or ion thrusters) are already a reality. The drive creates thrust by accelerating ions (charged particles) with electricity and has been used in the Deep Space 1 spacecraft that did a flyby of a nearby asteroid.

Although current ion drives does not provide blindingly fast acceleration (0-60 mph or 0-96 km/h in four days) the appeal to science fiction writers lay in the fact that the weight requirements for fuel are much lighter that traditional rockets. In theory it is also possible to ionize all elements known to man, so a spaceship could be build and fuelled on Mars, or Europa for example.

The concept of the ion drive has already been depicted as far back as 1910 by Donald W. Horner in his novel By Aeroplane to the Sun: Being the Adventures of a Daring Aviator and his Friends.

Another science fiction concept that is being taken seriously today is solar sails. Just as a normal ship use sails to harness the wind’s power, solar sails use the pressure of reflected solar radiation to propel the ship forward.

Jules Verne already mentioned the idea of light propelling spaceships in his 1865 novel From the Earth to the Moon although the science to explain this was not even available during that time. In 1951 an engineer named Carl Wiley wrote an article for Astounding Science Fiction (under a pseudonym Russel Saunders) called “Clipper Ships in Space”. This article influenced many science fiction writers, for example Pierre Boulle that mentioned these “sailcrafts” in his 1963 novel Planet of the Apes.

Finally, the most popular way of space travel used in science fiction today is of course the Hyperdrive, or faster-than-light (FTL) travel. Spaceships using this technology enters an alternative region in space, known as “hyper-space” by means of an energy field and exits at another location, thus performing travel that exceeds the speed of light.

Travelling via hyperspace, or “jumping” has been mentioned in short stories by Isaac Asimov (1940-1990), Arthur C. Clarke’s 1950 novel Technical Error, Star Trek, Star Wars as well as the Babylon 5 and Battlestar Galactica series.

Hyperspace might be a convenient way of bypassing Einstein’s Theory of Special Relativity stating that energy and mass are interchangeable, thus making speed of light travel impossible for material that weigh more than photons.

Conclusion

Space is big and even the closest star system to our own, Alpha Centauri, is 4.37 light years away.

With today’s technology the first humans that will reach Mars within the next two decades, have to endure a trip of 8.5 to 9 months just to get there. If we have any hope of reaching Alpha Centauri in one human’s lifetime, radical new ways of space travel must be invented.

And one science fiction writer’s dream today might just be the solution we were looking for all the time.

Technology behind the Panama Papers

Technology behind the Panama Papers

“Hello this is John Doe. Interested in some data?” This is how a reporter of the German newspaper Süddeutsche Zeitung was contacted in February 2015 via an encrypted chat service. This source would eventually leak 2.6 terabytes of information detailing how a Panamanian law firm helped clients to setup anonymous offshore companies and bank accounts.

This data was finally revealed to the world in April 2016 as the “Panama Papers” and the company implicated as Mossack Fonseca.

What happened in-between these two events are an almost cloak-and-dagger tale of the enormous effort by hundreds of investigative journalists, all made possible by the extensive use of technology.

John Doe

According to the German newspaper “the source wanted neither financial compensation nor anything else in return, apart from a few security measures.”  No personal meetings ever occurred and communication was always encrypted. He did indicate to the newspaper that his life was in danger.

It must be stated that, although it is not necessary illegal to have offshore bank accounts, many wealthy individuals and/or criminal organizations hide money in these accounts to prevent paying taxes in their own country. The purpose of this post is not to implicate any individuals, companies or firms that assisted or benefitted from the Mossack Fonseca scheme. It is purely a glimpse into the technology behind this leak – at least the bit that was publicly revealed.

Size of the leak

Over the next couple of months, the source would systematically release pieces of data to the German reporter. According to Süddeutsche Zeitung the amount of information shared totaled to:

  • Emails (4 804 618)
  • Database formats (3 047 306)
  • PDF documents (2 154 264)
  • Image files (1 117 026)
  • Text documents (320 166)
  • Other (2 242)

To put it into context: The Ashley Madison hack of 2015 was reported to be around 30 GB’s worth of data; the Sony Pictures leak of 2014 a massive 230 GB. The Panama Papers outweighs this by more than 10 times!

How the leak began

According to sources, the leak started as a fairly “normal” hack of Mossack Fonseca’s (MF) email servers. Also typical of these hacks were that MF was not open about it and quick to respond. As seen in so many of these cases this is partly in order to limit damage to a company’s public image, directors or boards not on par with the technical knowledge as well as insufficient technical staff to deal with it.

In addition, Forbes reported that the main MF portal clients used to access their sensitive information ran on a 3-year old version of Drupal 7.23, which had known vulnerabilities at the time that could be exploited by hackers.

So, while MF was ‘dealing’ with the situation, the anonymous source was syphoning off huge amounts of sensitive customer information and sending it to the German newspaper.

Please note that I am not promoting or glamorizing hacking at all. What I can say is that, even though hacking is illegal, there is a public tolerance towards it when a ‘bad’ company is hacked. Take the hacks of Ashley Madison vs. Sony Pictures – the former is perceived as a valiant act. Like Robin Hood stealing from the rich to give to the poor.

Finding needles in a giant haystack

The German newspaper appointed a five-person team that worked tiredly for two months to verify if the data was genuine.

It very soon became apparent that one of the major aims in the vast majority of cases were to conceal the true identities of the owners.

Trying to connect the dots in this web of complex secret transactions almost became and addiction to the team. “We often messaged each other at crazy times, like 2 a.m. or 4 a.m. about the newest findings” one of the reporters said.

But the sheer amount of data proved too much for this small investigative team. Imagine trying to find a cash receipt of a purchase made during a holiday 15 years ago and then cross-reference it to an email from a travel agency to confirm that you personally booked that holiday. Maybe not so difficult if you scanned and saved the receipt with a properly named filename and kept archives of your email conversations. Now try finding that information buried inside 2.6 TB’s worth of data without knowing any names up front nor that such a relationship should exist in the first place…

At the end, the newspaper could not sift through emails and account ledgers (covering nearly 30 years) on their own. They had to seek help and found it in the form of the International Consortium of Investigative Journalists.

Help from Down Under

The International Consortium of Investigative Journalists (ICIJ), is a non-profit organization based in Washington D.C. and has coordinated several previous projects that investigated financial data leaks.

Apart from Süddeutsche Zeitung the ICIJ invited many other influential newspapers and news agencies from across the world to form a coalition with the common goal of investigating the Panama Papers. This included Le Monde (France), The Guardian and the BBC (Britain) and La Nación (Argentina). Eventually 400 journalists joined forces in this effort.

From the start it was critical that the ICIJ investigation remained secret. But data still had to be shared between hundreds of journalists across the globe. In order to achieve this many software systems and packages had to be utilized – some open source as well as proprietary.

Journalists had to ensure that all files and their replicas were spread across different encrypted hard drives, using VeraCrypt software to lock up the information.

Süddeutsche Zeitung decided to use the software of an Australian-based company, called Nuix, which assisted the ICIJ in leak investigations before.

Nuix specializes in turning huge amounts of unstructured data into an indexed and searchable database. Its origins dates back to the year 2000 when a group of computer scientists at a Sydney university were exploring ways to process large amounts of data at high speed.

The newspaper journalists started by uploading the millions of documents to high-performance computers that were never connected to the internet in order to prevent the story from “breaking too early” or from those seeking to destroy it.

Once uploaded, they used optical character recognition (OCR) software to transform scanned images, such as ID documents or signed contracts, into human readable and searchable files. They could then start analyzing the data by applying searching algorithms provided by Nuix’s software. This allowed journalists to formulate questions that would in turn kick off the backend database search to look for matching data – exactly how web-based search engines work. The ability to index and analyze all types of data was the real key to the success behind the project.

Nuix actually stands for New Universal Intelligence Exchange engine, the name given to the software by the Australian computer scientists that developed it. The driving force behind Nuix was Jim McInerney who formed the company but passed away in 2004. Jim and his team originally started by processing email files on large scale, but they soon developed techniques to reverse engineer all major file formats, including some complex and proprietary ones like tiff images.

Just before Jim’s death, Nuix won a contract with the Australian Department of Defense. His family tried to run the company after his death, but eventually had to bring in a professional management team in 2006, led by new CEO, Eddie Sheehy that have worked with companies like Cisco before.

Some unexpected interest in their company resulted from the financial crisis of 2008.  A lot of money were lost in global financial markets due to the property bubble crash and people demanded answers. Software was supposed to help figure out what went wrong and who was to blame. Nuix became one of those solutions.

Nuix is certainly not the only firm that provides data processing solutions – even the ICIJ used other software as we will see later. Nuix made its name for being very fast in processing huge amounts of data.

Understanding files “at the level of ones and zeroes is what allows Nuix to achieve reliability and speed at scale” Eddie explains on their website.

The ICIJ’s technical army

The ICIJ is by no means just a group of ‘old-school’ investigative journalists. The have all the means and expertize required to operate in today’s digital and data-driven world.

Tools and software that were used in previous leak investigations were re-used or enhanced during this investigation. A lot of these tools were open-source.

Their search tool was based on Apache Solr and combined with Apache’s Tika, an indexing software that can also parse different file types like PDF.

They utilized a database search program called Blacklight that allowed the teams to hunt for specific names, countries or sources. On top of that there was also a real-time translation services for documents that were created in other languages. (Journalists primarily used English as the communication language).

Each news organization took their own precautions, restricting access to the secure computers that were used to connect to the ICIJ’s servers and ensuring that these were not accessible through their newsrooms’ regular networks.

The news broke

When the findings of the Panama Papers were released to the world on 3 April 2016, it immediately caught the public’s attention – mainly because some well-known and powerful individuals were implicated.

But although the story ensured some sensational newspaper headlines for many weeks it really was the hard work and collaboration effort of hundreds of individuals that worked behind the scenes that made it possible. All with the help of technology.

Sources

http://www.sueddeutsche.de

https://panamapapers.icij.org

http://www.nuix.com

http://www.forbes.com

http://www.theguardian.com

http://www.nytimes.com

https://en.wikipedia.org

Walking in Woods

Walking in Woods

In his 2006 book, “A Walk in The Woods – Rediscovering America on the Appalachian Trail”, Bill Bryson gives a delightful and humorous account on how he and his long-lost and former travelling partner, Stephen Katz, walked the Appalachian Trail.

Before reading this book, I have never heard of the Appalachian Trail. What really caught my attention was the shear length of it – approximately 3500 km’s (2200 miles) of marked hiking trail in the eastern United States between Springer Mountain (Georgia) in the south and Mount Katahdin (Maine) in the north.

In-between these two landmarks, the trail snakes through the states of North Carolina, Tennessee, Virginia, West Virginia, Maryland, Pennsylvania, New Jersey, New York, Connecticut, Massachusetts, Vermont and New Hampshire – 14 in total! One could expect that a trail of this magnitude might exists somewhere in Canada or the remote northern regions of Russia, but to find it in one of the highly developed and populated countries in the world is truly remarkable.

History

The idea of the Appalachian Trail (simply called the A.T.) was first proposed by Benton MacKaye in 1921 and in 1923 the first section of the trail was opened. MacKaye also called for a two-day conference to be held in 1925 and this meeting inspired the formation of the Appalachian Trail Conference, now called the Appalachian Trail Conservancy .

In 1930, a keen hiker and lawyer named Myron Avery took over the development work of the project and it is really his efforts that ensured the completion of the trail in 1937.

The trail and surrounding land is maintained and preserved today in a collaboration effort between government organizations, clubs and volunteers (like the Appalachian Trail Conservancy), which is the reason why it has endured for so many years.

Length

The actual length of the trail has changed over the years as new sections were added or existing sections re-routed. According to the Appalachian Trail Conservancy the latest official length has been established as 2181 miles (3509 km’s) in 2011.

If you look at the map below, you can appreciate its scale and grandeur. Some detailed maps tilt the trail on its side so that is displayed in a horizontal fashion from south to north, mostly because it is easier to pin it on a wall in this way!

appalachian-trail-map

Hiking

“Thru-hikers” are those who attempt to complete the trail in a single season. “Section-hikers” complete the trail in a serious of separate trips. The trail is usually hiked from south to north (Georgia to Maine) starting in March or April. A thru-hike requires anything from five to seven months, but hikers taking too long might have to abandon their effort when winter settles in causing cold, snowy and dangerous conditions.

Not everyone might get excited by the idea of putting on a rucksack to trod 3500 km through woodlands and camp outdoors. In fact, Bill could not find anyone to accompany him on his endeavor. At the last minute he received a phone call from his former travelling partner, Stephen Katz, excepting the invitation. Bill and Katz previously back-packed together through Europe as described in his 2001 journal, “Neither Here nor There: Travels in Europe”.

When his wife remarked that she thought they worked on each other’s nerves during their youthful European trip, Bill replied: “We started off on each other’s nerves. We ended up despising each other”.

Movie

A movie with the same title was released in 2015 starring Robert Redford as Bryson and Nick Nolte as Katz. Without the book as background, I felt that the movie did not really stood its own ground. The director probably had the same fear, hence the fact that seasoned actors, including Emma Thompson, were utilized to compensate for this. However, I can still recommend it as a good introduction to the A.T. and I know you would enjoy it if you love the outdoors.

Bears

In the movie there is a dramatic scene where Bill and Katz encountered a bear looking for food at their camp site. Although this is a bit of Hollywood entertainment thrown in, the chance of actual human-bear encounters on the trail are very rare. Even if you do run into one of these beasts, it would be the American black bear (Ursus americanus), not the dreaded (mainland) Grizzly bear (Ursus arctos horribilis). I cannot recall that Bill and Katz actually had any close encounter with a bear in the book. One night they might have heard some sniffing noises outside their tents, but it could have been any form of wild animal. Basic safety measures in any case dictate that you should not keep food in, or close to your tent but rather leave it in a tree outside your camping area.

Roads and Towns

Being located in a highly populated country like America, the trail is unfortunately crossed by many roads. The benefit is that it gives hikers the opportunity to hitch-hike or walk to the nearest town in order to indulge in some creature comforts if they desire.

There is a delightful scene in the book (not so well portrayed in the movie) where Bill describes his ill-fated effort to walk to the nearest Kmart in one of these towns to buy some insect-repellent. It turned out that the roads were not build for pedestrians at all – no sidewalks, pedestrian crossings etc. For most part he had to press himself against the dusty railings enduring angry looks from ignorant motorist who probably thought he was crazy. At one point, he decided it was safer to walk underneath a bridge spanning a small trickling creek than over it. When he finally arrived at the store, tired, rattled and covered in mud, he discovered that they did not sell any insect-repellent…

Conclusion

In case you are wondering, Bill and Katz never intended to do a “thru-hike”. At some point, they had to abandon the trail since Bill had work-related obligations to attend to. After that, Bill hiked little chunks of the trail on his own – sometimes just daily excursions. Bill and Katz reunited for a final time doing Maine’s infamous Hundred Mile Wilderness, but it proved to be the hardest part of the trail so far. In the end, after some near misfortune, they decided to stop pretending to be mountain men and headed home.

But as Katz said, after they hitched a ride into the nearest town and ordered some cream sodas: “As far as I’m concerned, I hiked the Appalachian Trail. I hiked it in snow and I hiked it in heat. I hiked it in the South and I hiked in the North. I hiked it till my feet bled. I hiked the Appalachian Trail, Bryson.”

Retro Technology

Technology, computers and programming have always been a major part of my life. It also provided me with an exciting career so far; recently in the Business Intelligence, SQL and databases environment. I think that most technology workers and people in the IT industry can feel blessed that they are pursuing careers doing something they really enjoy.

However, it is not just new technology that excites me. The history and development of it is also fascinating.

In this post, I am going to look back at some of the “retro” technologies that sparked my love of computers and programming in the first place. As with most of these type of stories, it started with the boy that lived down the street…

Punch Cards

This boy’s father worked at an insurance company that used punch cards. He occasionally brought some of them home and we soon discovered that these stiff little pieces of paper had multiple purposes – from building card-houses to sticking them in your bicycle’s wheels to make a noise pretending that you were driving a motor cycle!

punch-card
Punch Card (www-03.ibm.com)

Even though we knew that they were somehow associated with “computers” we really had no idea what their purpose was. It was many years later that I only learned that these cards actually represented information by the presence (or absence) of holes in predefines positions.

So they were in essence computer files or programs stored on paper instead of magnetic disks and could be used to automate data input or directly control automated machinery.

 

ZX Spectrum

This boy was also the first in the street to own a “home computer” called the Sinclair ZX Spectrum.

zx-spectrum
ZX Spectrum (www.myprius.co.za)

The Spectrum was an 8-bit computer developed by Sinclair Research Ltd in 1982 and manufactured in Scotland. It could plug into a normal CRT television set and programs were stored on magnetic tapes that were loaded via a peripheral tape deck. The name “Spectrum” as well as the rainbow image on the keyboard were chosen to highlight the machine’s colour display capabilities.

I can recall how we had to wait for the tapes to forward or rewind when loading a new game or program, patiently watching the analogue tape counter ticking over. It felt like hours!

zx-spectrum-with-casette
ZX Spectrum with cassette (www.overtakenbyevents.com)

The Spectrum was the first mainstream home computer and it paved the way for other brands like the Commodore 64, BBC Microcomputers and ultimately the IBM PC to find its way into living rooms and studies.

 

BBC Micro (computer)

The BBC Micro was my first home computer. These computers (and associated peripherals) were designed in the early 1980’s by the Acorn Computer Company for the BBC Computer Literacy Project.

bbc-micro
BBC Micro (www.wikipedia.org)

Due to its roots as an educational tool, BBC Micro’s were used by many schools in the United Kingdom. My primary school (in South Africa) also purchased a couple of them and basic computer literacy classes where given to the older students on Fridays. There was also a small computer club that came together every week for more advanced programming lessons.

The RAM (Random Access Memory) was minute in todays’ terms (16 KB for the Model A and 32 KB for the Model B) and it did not contain any build-in hard drives. The floppy drive was also connected as a peripheral device for loading programs and games.

Four 16 KB ROM (Read-Only Memory) sockets where provided that could be used for ROM chips containing program languages and applications. This helped freeing up most of the RAM to be used for program execution.

My computer contained a ROM chip with a word processing application as well as one for the BBC BASIC programming language (BASIC stands for Beginner’s All-Purpose Symbolic Instruction Code). BASIC was an easy to learn language that provided a good platform for developers that later moved onto other languages like PASCAL, C, C++ or Visual Basic.

 

Logo

I wrote my first computer program using the Logo programming language.

Logo (designed in the late 1960’s) was synonymous with the “turtle” or triangle cursor that graphically shown the output of your computer program on the screen, typically line graphics.

logo-program
Logo Programming (www.theantiroom.files.wordpress.com)

A square could be drawn with the following command sequence: FD 100, RT 90, FD 100, RT 90, FD 100, RT 90 and FD 100. The commands necessary to draw the square could then be stored together as a ‘function’ that could be called multiple times. By turning the turtle, say 15 degrees (RT 15) between each function call would draw a complicated image that represented something like a flower.

It must be noted that “Logo” is not an acronym (like BASIC) and was derived from the Greek logos meaning “word” or “thought” to distinguish itself from other languages that were not graphically oriented.

My school also had a “real” turtle that was connected as a peripheral device to the computer. A marker pen was attached to the turtle’s body and it could then draw the output of your program on a piece of paper on the floor.

logo-turtle
Logo Turtle

 

Games

It cannot be denied that many programmers, like myself, got involved with computers due to the interest in games. At some point you start asking yourself how a game is actually written and if it is possible to create one yourself.

Our game of choice was called “Chuckie Egg” and the objective was to collect twelve yellow eggs positioned in each level before a countdown timer reached zero. Bumping into little “chicks” instantly killed you, but they followed predictive routes and stopped to feed on the purple piles of seeds. When your player ate the seeds in return, it temporary slowed down the timer.

In later levels the “mother hen” was released from her cage and she could fly through any obstacles (ladders, platforms) heading directly for your player. She was, however, slow to turn around and you could use it to your advantage to draw her away from your eggs.

chucky-egg-game
Chuckie Egg Game (www.repton3.co.uk)

Today this game can easily be recreated with game making software like Game Maker Studio, but one can appreciate the programming effort – bearing in mind that the executable file was typically stored on a 360 KB floppy disk and ran on a computer with 16 KB RAM, like the BBC Micro.

You can try out this game online at the following site: http://www.repton3.co.uk/chuckieegg.aspx

 

Conclusion

To conclude, the history of technology and computers always fascinates me. I hope you enjoyed this post and that it evoked some of your own childhood memories playing Mario Brothers or creating your first computer program!

2017 – A New Year

We are almost halfway through the first month of 2017. The yearly family holiday a distant memory; New Year’s Resolutions already broken and forgotten…

Even if you do not follow the tradition of ‘New Year’s Resolutions’ most people use the start of a new year to reflect on the year that has been and set personal and career goals for the year ahead.

But why is it that a new year evoke such deep philosophical emotions within human beings? After all, in astronomical terms there is hardly any difference between the 31st of December and the 1st of January. In the Southern Hemisphere the day is slightly shorter and the night slightly longer, but most humans probably don’t even take notice.

I think the answer lies in the promise of excitement and change that a new year brings.

In ‘The Screwtape Letters’, C.S. Lewis has given us some insight into this:

“He [God] has balance the love of change in them [humans] by love of permanence. He has contrived to gratify both tastes together in the very world He has made, by that union of change and permanence which we call Rhythm. He gives them the seasons, each season different yet every year the same, so that spring is always felt as a novelty yet always as the recurrence of an immemorial theme.”

“The pleasure of novelty is by its very nature more subject than any other law of diminishing returns.”

We humans are wired to respond emotionally to change, albeit positively or negatively.

May 2017 provide you with a positive experience and may you achieve all your personal and career goals!