Introducing notebookJS: seamless integration between Python and JavaScript in Computational Notebooks
Have you ever seen a data visualization and thought it would be great to use it in a Jupyter Notebook? I know I have, many times. Notebooks are the perfect place to experiment with data visualization, because they can contain all the data wrangling, preprocessing, pictures, and documentation in the same place. However, adding JavaScript to a Python notebook requires a lot of boilerplate code, which can become cumbersome and repetitive. To make matters worse, different Notebook environments have incompatible APIs for message passing between Python and JavaScript. For example, if a developer wants their library to run in both Jupyter and Colab, they will have to use two communication formats.
To address these issues, we created notebookJS, a Python library that enables the integration of Python and JavaScript with a single line of code.
Background
Writing a visualization in your favorite web stack (D3 and React for me ❤) provides a lot of flexibility to the developer. However, testing the code on different datasets can become a challenge. In my experience, I’ve had to either 1) hard-code datasets, or 2) write a custom server to manage the data. None of these options are very good when experimenting with datasets, preprocessing steps, or chart types.
Experimenting is easy in Python Notebooks, since all the data manipulation can be done inline and the notebook automatically keeps track of the computational steps. In most cases, Python libraries such as Matplotlib and Altair can be readily used to create charts. However, there are times when the visualization we need cannot be generated with out-of-the-box tools. For example, when I wanted to explore ML pipelines produced by Auto-Sklearn, I had to write a visualization from scratch. Integrating custom visualizations in notebooks can be cumbersome because there’s a lot of boilerplate code needed to 1) run the visualization inside the notebook and 2) send data between Python and JavaScript (see this paper [1] for details).
Since we find visualizations in Notebooks so useful, we decided to create a library for it. In particular, notebookJS:
- Takes care of all the boilerplate code for vis in notebooks;
- Automatically loads JS libraries from the web
- Supports code split into multiple JS and CSS files.
Here’s how it works:
notebookJS
First of all, you have to install the library:
pip install notebookjsThe notebookJS API is very simple. Actually, a single function takes care of everything: the execute_js method executes a JavaScript function and sets up the infrastructure for bidirectional communication between Python and JavaScript using callbacks.
These are the parameters:
- library_list : list of str. List of strings containing either 1) URL to a JavaScript library, 2) JavaScript code, 3) JavaScript bundle
- main_function : str. Name of the main function to be called. The function will be called with two parameters: <div_id>, for example “#my_div”, and <data_dict>.
- data_dict : dict. Dictionary containing the data to be passed to <main_function>
- callbacks : dict. Dictionary of the form {<callback_str_id> : <python_function>}. The JavaScript library can use callbacks to talk to python.
- css_list : list of str. List of strings containing either 1) URL to a CSS stylesheet or 2) CSS styles
The main_function is the JavaScript function that will be run when execute_js is called. It has the following signature:
function main_function(div_id, data_dict)As a simple example, we can use D3 to add a circle to the output cell:
Loading local libraries
As we have seen, running a JavaScript function in the Python Notebook takes a single line of code (excluding JS library definitions). We note that the JS code can be stored in a separate file and loaded from Python. Data can also be loaded in Python and passed to JavaScript using a dictionary structure (internally converted to JSON).
For example, if we want to reuse this radial bar chart, we just have to change the data loading code in the JS file and then use notebookJS:
Python Callbacks
One of my favorite features of notebookJS is the ability to set up Python Callbacks. When working on data science projects, some steps are more easily done in JavaScript (like front-end and user interaction), and some are better in Python (number crunching, machine learning, etc). With notebookJS, we can use the best of both worlds.
Let’s see a very simple example: we want to make an animation where we write Hello World in a bunch of languages. The phrases are stored in Python, and Javascript handles the front-end code:
The Javascript drawing function asks Python for “Hello World” every 1 second using the identifier “get_hello”:
Python waits for get_hello and responds accordingly:
The callback function is “connected” to the visualization using the callbacks parameter:
Limitations
notebookJS currently does not support Javascript ES6 modules. This is the reason why we are using D3 V3 in our examples.
If you need ES6, we recommend using a build tool such as Webpack + Babel to compile your JS code to a single file and transform the ES6 to old JS. As projects grow, this is the easiest way to manage multiple libraries and create optimized code. For an example of code bundling, see the Webpack library setup on this page.
Final Thoughts
notebookJS is a new tool and we are actively seeking feedback on what features we should support. Let us know if you have any questions or would like to collaborate!
We have more examples available in our repository. For a live demo, please see our Colab notebook.
References
[1] Jorge Piazentin Ono, Juliana Freire, and Claudio T. Silva. “Interactive Data Visualization in Jupyter Notebooks.” Computing in Science & Engineering 23.2 (2021): 99–106.
