The usual way to get data is to ask someone for it. But there are two unusual sources of data: you can buy it, and you can salvage it.
SALVAGEDATA is a good way to get started. It means getting data that is already available, but in a form too clumsy or tedious to use. For example, the Census Bureau publishes a huge amount of data as part of the Statistical Abstract of the United States. That’s great for social scientists, but if you want to make a map with the data, you have to do a fair amount of work yourself.
Another example: take all the books out of your house and put them on the floor to form a path from your front door to your bedroom. Then walk along this path blindfolded from front door to bedroom without touching any books. This is an example of what computer scientists call “salvaged data,” because although it is not in digital form, it can be readily converted into digital form by taking pictures and using optical character recognition software (OCR).
You can’t salvage data. The best you can do is copy it.
We have to make a distinction between “data” and “information”. Information is stuff we understand or use. Data is just a bunch of bits. You can have data without information, but not information without data. So if you want to make information you must first get your hands on some data.
The most common way people get data is by copying it from someone else. If you get data from another source that means they had it first, which means they copied it from someone also, and so on. In the end there has to be a first source of any piece of information, which means there must be someone in the world who took the trouble to produce it instead of just copying it from someone else. This seems obvious when you think about it, but it’s amazing how often people overlook this fact.
People who want to get rich by starting companies are advised to look for problems, preferably problems you have yourself. But this is wrong, or at least incomplete: you should look for problems that cause other people pain, because those are the only ones worth solving.
If you’re trying to decide whether to work on an idea, one of the most decisive tests is:
The two most common data disasters are losing your data and losing your ability to use it. Data is like a car: it’s easy to forget about when it’s working well, but you’re completely stuck if anything goes wrong.
There’s no excuse for not planning for data disasters. The risk is too great. This is something you can do on your own, by keeping backups of your data and by using open file formats.
The most obvious way to preserve your data is to make lots of copies, and put them in different places. I have four copies of my recent book (including the manuscript in two different formats). One copy is at home, one at work, one on my laptop, and one online.
If I lost any one of these copies I could recover from the others. If I ever had to move a copy, I could do so with confidence that all the other copies would still be there when I got back.
