feat: allow loading nodes from env-specified paths

[jmorahan]

This allows you to specify additional paths in an environment variable so you can maintain custom components outside of the core codebase

[FintanGalvin]

Fantastic pr

[toi500]

Best PR I have seen from the open-source community for this project so far. Love it.

[HenryHengZJ]

awesome addition! curious how this work? you need to build the component files as well right? in the COMPONENTS_PATH?

[jmorahan]

awesome addition! curious how this work? you need to build the component files as well right? in the COMPONENTS_PATH?

That is the idea, yes.

I haven't set up anything to facilitate that in this PR, as I figured it would need more discussion; this just sets it up to work if you can get the components built. But I did do a simple textsplitter component locally to test; can share that if it would be helpful. Just cloned the relevant bits of the interfaces etc for now.

[toi500]

@jmorahan I’ve been testing this out and wanted to highlight 2 points:

• I needed to make a small modification so the patch works on Windows too (using path.delimiter for splitting COMPONENTS_PATH).

• I noticed that for a custom node to appear, it must have a unique this.name value. If I use the same this.name as an existing node, the custom node does not show up (the original is loaded). So, as far as I can tell, there isn’t currently a way to override existing nodes, only to add new ones with unique names.

And here is the thing, if I’m understanding correctly, when we want to make a simple change to a node like the Loop node, we need to change its default this.name value so it shows up, making the node unoperational.

It would be very helpful if external nodes with the same system names could override the native node by default.

[toi500]

Following my above comment, I have added the ability to override existing nodes, as the more optimal way to work with this new feature. Now, we can create nodes and components from scratch using unique this.name values, or override existing ones when needed (by using the same this.name values as the default node) , for example, if a small change is required in a particular node.

Changes: bd17c55