from Hacker News

Node.js and Python Interoperability

by fridgamarator on 6/16/19, 3:25 AM with 21 comments

  • by swordfeng on 6/16/19, 8:48 AM

    I was once trying to do something even crazier: exposing full python accessibility to node.js, using proxies. So that you can use python objects and functions as native javascript objects and functions. It is feasible because the languages and runtimes share a number of common designs.

    There were several tough things however. The first was error handling. Yes you can catch exceptions in native code and convert it to exceptions in the other language, but it's not straightforward to keep stacktraces. The second was circular references between runtimes. Since references across boundaries are global, garbage collector on either side could not reclaim circularly referenced objects. Although this could be resolved by manually breaking up the circle, it could be better to have weak references. (Or maybe other utilities, idk what would be more elegant.) The third was that js has no operator overloading, so I had to use .__add__() for example to call the python add operator.

    One line example: https://github.com/swordfeng/pyjs/blob/master/test/jsobject.... It's a toy project I did years ago and not even compiling now. Also I was wondering if anyone really need to do things in this way, given there are bunch of popular and stable RPC libraries. But I was happy to learn something about underlying cpython and v8 from it.

  • by rkeene2 on 6/16/19, 5:08 AM

    I recently did something similar with Tcl (instead of Python) and Duktape (instead of V8): https://rkeene.dev/js-repl/

    All the Tcl runtime is in the "runtime" object, so like "runtime.puts('Hello World')" or "runtime.expr('2128')", etc

    It was a lot of fun

  • by lmeyerov on 6/16/19, 4:17 PM

    We needed to solve this as part of using pydata/GPU service calls from our node app. While we do have a couple of native modules, they're a PITA. Our solution for more generic code is async HTTP service calls passing typed Apache Arrow tables ( https://arrow.apache.org/docs/js/ ), which gives a cleaner path to maintenance, observability, packaging, distribution, low overhead etc.

    The current trick we're looking at is making this zero-copy when same-node, esp for GPU code, so happy to chat with folks about that!

  • by snek on 6/16/19, 1:38 PM

    Very cool idea! For all of you thinking of writing native modules for node, please remember to use napi, not nan. napi is fully abi stable, while nan is not!

    https://nodejs.org/api/n-api.html

    We also have a c++ wrapper: https://github.com/nodejs/node-addon-api

  • by santa_boy on 6/16/19, 7:26 AM

    I'm having the requirement to work with Node and Python now. I write my webapps in node but have data analytics scripts written in python to be invoked. I'm planning to use [python-shell - npm](https://www.npmjs.com/package/python-shell)

  • by fermigier on 6/16/19, 11:10 AM

    Also: https://github.com/bobpepin/pyduktape (interop between the Duktape JS interpreter - https://duktape.org/ - and Python).

  • by rcfox on 6/16/19, 7:02 AM

    I've worked the opposite way: calling Javascript from Python, using PyV8. It worked okay enough, but it felt super fragile. We only used it to run a tiny bit of client-side code on the server. I wouldn't want to have a large project where Javascript and Python interact heavily.