Skip to main content

Set Up Nightly File Uploads

The LifeOmic Platform allows its users to upload files via the LifeOmic CLI. There are multiple ways to authenticate with the CLI, but for this document we’ll be using API keys. Once an API key is generated, the CLI can be used in conjunction with a cron job to automatically upload files each night.

To generate an API key, complete the Set up API keys procedure.

Download and verify CLI​

wget "https://github.com/lifeomic/cli/releases/download/v6.11.0/lo-linux-x64.zip"
sudo apt-get install unzip
unzip lo-linux-x64.zip
rm lo-linux-x64.zip
sudo mv lo /usr/local/bin
lo --help

Setup CLI with API Key​

lo setup
? Pick a LifeOmic environment: prod-us
? Specify a default LifeOmic account to use: myaccount
? Use API key for authentication? Yes
? API Key:

After pasting the API Key, you should be ready to use the CLI for file uploading. Verify that your authentication is working with the following command:

lo projects list

Prepare local folders and the LifeOmic Platform project​

mkdir uploads
mkdir logs
lo projects create "Files Test"

Replace "Files Test" with whatever the project should be called.

Prepare upload script​

Copy project ID

lo projects list
Example response:
items:
  -
    id:   f3a220a0-b22b-41c3-9f77-2ce7cf37c93b
    name: Files Test

Copy the ID of the project you created.

Create script​

vi lo-upload.sh
#!/bin/bash

set -u
set -e
set -x

if pgrep --full "lo files upload" ; then
        echo "Already running"
        exit 0
fi

trap "echo Process killed" SIGINT SIGTERM SIGABRT

echo `date '+%Y-%m-%d %H:%M:%S'`
cd /home/ubuntu/uploads
/usr/local/bin/lo files upload ./ f3a220a0-b22b-41c3-9f77-2ce7cf37c93b --recursive --overwrite --delete-after-upload
note

In this example we’re using /home/ubuntu/uploads. You may need to change that line of the script to match the directory you’re using.

note

Make sure to replace the UUID (which starts with f3a220a0-) above with your project ID.

Add execution permissions and verify a file upload​

chmod +x lo-upload.sh
echo "test" > uploads/test.txt
./lo-upload.sh

Verify you see output similar to this:

Upload complete: test.txt, ID: `d4d9d8fc-b96b-49ad-a43c-c24f54b29941`

Also, the following command should display the uploaded file:

lo files list <projectId>

Create a cron job to execute the script​

sudo mv lo-upload.sh /etc/cron.daily/
crontab -e

Add the following line (the script will run nightly at midnight) and then save:

0 0 * * * /usr/bin/timeout -s 2 172800 /etc/cron.daily/lo-upload.sh >> /home/ubuntu/logs/lo-upload.log 2>&1

Email success or failure​

In addition to sending output to the upload.log file as shown above, you can receive nightly emails to monitor success or failure.

Configure SMTP​

You will need to configure your server to be able to send email. One tool commonly used for this purpose is postfix. For postfix installation and configuration, search the web for setup instructions specific to the type of SMTP server you’ll be using (Office 365, Gmail, and others).

Make sure to test an example email before moving on from this step:

sudo apt-get install mailutils
echo "Test email contents" | mail -s "Test email subject" "email1@domain.com,email2@domain.com"

Reconfigure cron job to email​

Update your crontab -e entry to use mail:

0 0 * * * /usr/bin/timeout -s 2 172800 /etc/cron.daily/lo-upload.sh 2>&1 | mail -s "LifeOmic Platform <companyName> Files Upload" "email1@domain.com,email2@domain.com"