Using Timemap for Cartographic Platforms

Timemap is an open source software to visualise geospatial events in an interactive platform. We’re constantly updating the codebase to support new features and data types, but the current version will end up looking something like our platform documenting Russian military presence in Eastern Ukraine. We developed Timemap to allow us to dynamically create new platforms from a set of events in either a Google or Excel spreadsheet. We use timemap to present research findings online and in exhibitions; but we also use it dynamically throughout all stages of an investigation, and hope that other open source journalists and research agencies will find it similarly useful. The density or scarcity of events in the axes of time and space informs where we ought to next focus our research, where and when data resolution is high, and where and when we need more moments.

This tutorial will walk you through setting up a timemap instance using data in an example Google Sheet, which is already formatted appropriately to serve such an interactive application. Once you have this demo working, creating your own interactive platform is as easy as modifying the data in the sheet and reloading. Here we’ll configure the application using a locally stored Excel spreadsheet (which is included in the default datasheet-server download), but there is an appendix indicating how you can configure timemap to read dynamic data from a live Google spreadsheet with a few additional steps. (Note that if you are looking for a platform that includes videos and images, these will need to be hosted separately, and the appropriate URLs referenced in your data sheet.)

Take a moment to look over the structure of the data in the example sheet. There are comments in the sheet regarding the role of individual tabs and cells. (Hover over the cells with yellow tags in the top right corner.) A basic timemap instance requires a sheet structured in a specific way, so that timemap can do its work.

The tabs in a timemap data sheet are separated into two distinct types: source tabs and export tabs.

• Source tabs – Source tabs are not directly used by timemap, and so can be structured in whatever way makes the most sense for your investigation. You’ll spend most of your time in these source tabs, adding and editing, and they’ll often contain additional data that isn’t necessarily visualised in timemap. There is no essential format for source tabs, and they should be designed primarily with human/researcher readability and workability.
• Export tabs – Export tabs reference data in source tabs, and format it into rows and columns so that it can be displayed in a timemap instance. As a matter of good practice, we only place referential data in export tabs. In other words, cells in export tabs consist entirely of the Excel-like expressions that Google Sheets offers to populate a cell with data from elsewhere in the sheet. See the comments in the EXPORT_EVENTS tabs for more information and example of how referential data works in Google Sheets.

However you structure your source tabs, keep in mind that in order for timemap to work, you ultimately need to be able to extract events into the export tabs. Events are the main data type in timemap, and understanding how they work is crucial to working effectively with timemap.

An ‘event’ in timemap is something very specific: a distinct moment in time and/or space that is constructed on the basis on one or more pieces of evidence. When working with timemap, it is crucial to think about events in this very specific way, and not in the more generic way that we can talk about events in general.

Let’s look at some examples of events in the example sheet to get familiar with the concept. The most standard event is something that happens at a distinct moment, in a distinct space, such as the first row in the sheet: “Person A appears on the scene.” The event in the next row is an event that occurs in the same place, but at a different time, and is thus codified as a separate event: “Person A begins recording.”

Information often need to be ‘broken down’ in this way, into distinct space/time moments, in order to be visualized effectively in timemap. The question you should be asking yourself when considering whether information should be entered as a separate event is; “is this information regarding a separate moment in space or time?”

In order to retain lateral associations between atomic events as you break them down, you should use filters. You can think about filters as tags that add a label to an event, so that it can be associated with other atomic events. In the example sheet, the events in rows 2 and 3 both relate to Person A: so we can add a ‘Person A’ filter to both events. In timemap, this will appear as a filter that can be applied to see only a certain subset of events: i.e. all those that pertain to the narrative of Person A.

For more information regarding events, filters, and other types, don’t hesitate to ask on our Discord server.

This section will demonstrate how to download the source code to up a local timemap instance that is connected to the example sheet we’re now familiar with. Once the code is installed, you can easily modify or add new sheets by changing a few lines in config files. Note that this section assumes several things:

1. You are using a bash shell. If you are on Mac or Linux, this will be your default. If you are using Windows, install Git Bash before continuing.
2. You have Git installed and ready to use on your computer. If not, follow the instructions here.
3. You have Node on your computer. This is a web development suite that allow you to build and run websites locally. You can install Node from its official website.

Now you’ve got bash, git and Node, you’re ready to go. We need to download and install datasheet-server. Datasheet-server is a simple server that proxies the data in the example sheet, making it available as JSON endpoints. Clone and install dependencies with the following commands in a new bash terminal. Each line is a new command, so you should run three commands (one for each line). If you are using Windows, run these commands in Git Bash.


git clone https://github.com/forensic-architecture/datasheet-server
cd datasheet-server
npm install


If one of these commands returns an error, it’s likely that you haven’t properly installed one of the dependencies listed above.

If the last command runs to completion, you should have datasheet-server installed and ready to run. Inside the ‘datasheet-server’ folder that was downloaded, open up the timemap_data.xlsx file in the ‘data’ folder. This file is an XLSX equivalent of the example Google sheet, so it should look familiar. This is the data that we will turn into a timemap.

In the same terminal, run the following command to spin up the server:


npm run dev


You should see something similar to the following output:


Enabling CORS in development...
Connected to timemap_data. No explicit authentication required for XLSX Files.
===================
Started on port 4040


To check that the data is available, visit http://localhost:4040/api/blueprints. You should see a page indicating that the server hasn’t got any data in it yet, with an UPDATE button in the top right-hand corner. This is your data management panel; changes that you make in the sheet won’t immediately be reflected in timemap, as datasheet-server sits as an intermediary. Once you’re happy with your changes, visit this page and hit the UPDATE button, and your changes will propagate through to timemap.

Hit the UPDATE button now to load the data from the default sheet. This should return the following success message:


{"success":"All sheets updated"}


If you now reload the blueprints page, you should see that the server is now populated with several endpoints. This indicates that your data is loaded and ready to be displayed in timemap!

IMPORTANT: when you are finished working with datasheet-server, ensure to tap Ctrl-C in the command line in order to kill the process. Otherwise you may find yourself with a hanging process next time you try to start it.

Now that datasheet-server is running with data loaded, we are ready to download timemap to view it interactively. Keeping that terminal running, open a new window in your terminal, and navigate to the same parent folder that contains datasheet-server. Once there, run the following commands:


git clone https://github.com/forensic-architecture/timemap
cd timemap
cp example.config.js config.js
npm install
npm run dev


These commands may take a few minutes, and the last command (npm run dev) will hang on your terminal with a message saying “Compiled with warnings.”

Once you see this message, you should be able to visit http://localhost:8080 to see a local instance of timemap that presents the data from the XLSX sheet. If you get errors, make sure that your local instance of datasheet-server is still running at http://localhost:4040/api/blueprints, and that timemap is also running in a separate terminal.

Congratulations! You have a local instance of timemap running, and you’re ready to get started with your investigation. Look out for another post soon that details more of the practical ways in which we work with timemap throughout investigations.

Datasheet-server can be connected to load data from a Google spreadsheet, rather than a local Excel file. In order to set this up, you’ll need to create an account on Google Cloud Platform. Once you have an account, walk through the following steps in Google Cloud Platform (GCP).

1. Create a new project.
2. In the new project, enable the Sheets API.
3. From the project’s credentials page, create a new service account key.

After you’ve created the service account, you should download the JSON credentials onto your local machine.

The downloaded ‘credentials.json’ contains information for your service account such its associated email and its private key. Inside the datasheet-server folder you cloned earlier, copy the env.example file in the top layer to a file named ‘.env’ that is also in the root directory. Open up this file and replace the value in the double quotes in the lines that begin with SERVICE_ACCOUNT_EMAIL and SERVICE_ACCOUNT_PRIVATE_KEY with the values ‘client_email’ and ‘private_key’ (in ‘credential.json’) respectively.

Inside the ‘src’ folder, open up the ‘config.js’ file in a text editor. Replace it with the following:


import { tabs, prefixedTabs } from './lib'

export default {
  gsheets: [
    {
      name: 'example',
      id: '1UC7DkCFeUXHfpUxUGruExwFbP4pqVBdJLOKfo6wDDGk',
      tabs
    }
  ],
  xlsx: [
    {
      name: 'timemap_data',
      path: 'data/timemap_data.xlsx',
      tabs
    }
  ]
}


Now restart the server (by hitting Ctrl-C in the terminal running the process from before, and then again executing npm run dev). You should now see the following output:


Enabling CORS in development...
Connected to 1UC7DkCFeUXHfpUxUGruExwFbP4pqVBdJLOKfo6wDDGk.
===================
grant access to: {{your service account email}}
===================
Started on port 4040


Note that when you are adjusting datasheet-server to point to a private Google Sheet, the update request will not go through unless you add your service account’s email address to the sheet with at least View permissions (through the ‘Share’ button in the top right).