Getting Started #

Welcome to 1nb documentation.

In this doc we are going to explain how you can incorporate 1nb to your python projects. First, you need to install the wheel (download) with pip:

pip install <downloaded.whl file> --user

We recommend installing it in the user prefix so that you can use it for all projects.

Initialize a repository #

7nb init --name <repository name> --requirements-file <pip requirements file>

Executing this in the git root of a repository will setup the necessary structures (in /.7nb) and make it ready for use. Doing this step is not mandatory. However, doing this will set the correct name for the repo and not rely on guessing from the origin url.

The requirements-file option allows 1nb to know which packages are required to execute the notebooks in this repo. This file will be watched so that on change, the updates will be automatically recorded as well.

Local packages

If there is local python code that the notebooks depend on, those can be included as well. The code needs to be packaged in setuptools style and a pyproject.toml needs to be created so that the package is pip installable.

Finally add the package path to your requirements file mentioned above: ./my/project/path

Saving data #

from SevenNB.exports import MyData

This code creates a data export named MyData. Writing data to it is similar to using the open syntax for saving binary data to disk:

with MyData.open(suffix=".npy") as out:
    np.save(array, out)

More examples:

import pickle
from SevenNB.exports import labels

vocab = {...}

with labels.open(suffix=".pkl", mode='wb') as fe:
    pickle.dump(vocab, fe)

import pandas as pd
from SevenNB.exports import Sales

out = pd.DataFrame(...)

with Sales.open(suffix=".csv") as f:
    out.to_csv(f)

Suffix is required to identify the type of data being saved. To save text data change the mode to text: .open(mode="t", suffix=".json").

This code can be run from any script or notebook as many times. The name of the export MyData will be an unique identifier. So exporting with the same name from other locations will overwrite it.

Parameterize your data #

Data that you save to 1nb can be parameterized. These parameters will be later used for retrieval as well.

myparams = {"year": 2023, "foo": "bar"}

from SevenNB.exports._myparams import Report

Report.save_as_json({"sales": ...})

Here the data saved to Report will have its parameters set to the key-values in myparams. If this code is executed multiple times while changing the parameters, every data blob will be saved against their corresponding parameter set in Report.

Parameterize your notebooks #

The parameters in a notebook should additionally be defined in a cell that is tagged parameters. This cell should be the first cell of the notebook:

year = 2023
foo = "bar"

Later:

p1 = dict(year=year, foo=foo)
from SevenNB.exports._p1 import Report

Defining the parameters in this way allows easily modifiying key assumptions while executing the notebook. The parameters values should be simple literals and not include objects.

Also, if the data values are simple (boolean, string, number) then 1nb will allow the user to modify it in the web UI while executing it later.

Warning: you should include all the variables in the parameters cell to the import definition too (p1). Failing to do so will result in errors in the push stage.

Result from notebooks #

Often a notebook has some key cells that have the final result in them. It maybe some charts or text data. Tagging these cells as result will make the output from those cells easily viewable in the web ui. Code from these cells will not be shown.

Adding notebooks #

After executing your notebook with a parameter set you should add it to 1nb:

7nb addnb --nb <notebook.ipynb>

This operation will identify the parameters in the notebook and save it for later upload. This operation needs to be done every time you have a notebook ready with a unique parameter set.

In case this notebook is not the one that was originally executed (e.g. it was renamed or moved after execution) then please add the original one with --original. In that case --nb should be the notebook that was saved after execute:

7nb addnb --nb <notebook.ipynb> --original <original.notebook.ipynb>

Upload #

Before your work can be uploaded, it first needs to be commited and tagged to git:

git commit
git tag reportsv1

7nb push

On performing 7nb push all the data exports and notebooks are going to be uploaded to the cloud. If this is the first push, use 7nb push --create instead. After this you should be able to view this repository in 1nb.ai

Importing data #

Data can be loaded with syntax similar to exporting, but with the tag/commit info to identify the revision version of the code that exported it. It then needs to be read as usual in python, but using .reader() wherever a file handle is expected :

from SevenNB.i.<repository>.<tag> import MyData

# example: read csv
import pandas as pd
my_data = pd.read_csv(MyData.reader())

# example: read pickle
import pickle
my_data = pickle.load(MyData.reader())

# parameterized data
p = {"year": 2023, "foo": "bar"}
from SevenNB.i.<repository>.<tag>._p import Report

Exporting data from such a script/notebook will produce a dependency on the exports MyData and Report. Viewing the export in the web UI will list these dependancies.

Also, if this is a notebook and the dependant exports are also generated by notebooks, 1nb will automatically create a pipeline such that the dependant notebooks are rerun (if required) before reunning this notebook.

In case the repository name has special characters so that python throws SyntaxError, replace those characters with an underscore (_).