When I was at the University, I wrote a programme to analyse data collected by a radar. My programme was quite slow. It needed 30 hours to generate just one view of the data.
30 hours of processing time of an IBM mainframe are expensive, very expensive. Happily I could rewrite the programme making it faster. It needed only one hour after the change.
The language used was the exotic APL. You need terminals with special keyboards to write software in APL. It’s like a set of mathematical symbols. This was making things more difficult.While creating a system intended to send production orders to an industrial plant, I needed to write my own database application because the ones available at that time were too slow.
I had to write my own algorithms to manage tables, build and rebuild indexes, search and write information.
Creating a data-warehouse which purpose was to study billions of telephonic calls, I had to be very careful not to write slow queries. Every one of them was lasting many hours. Some of them were lasting for a whole day or two. I had to optimise them carefully because we had “only” a month to provide the final result.
One of them went from 20-30 hours of execution time to 30 minutes.
I have written a server process to collect information from thousands of computers and form a distributed database. The information was about events and you could either slow down the event influx or put them in a queue. The process had to be very fast in order for it not to lose events. The server process was part of a project for Intel Corporation.