Multi-file support RFC

[hatemhosny]

LiveCodes does not currently support a file system. The (compiled) code from the 3 editors (markup, style & script) are combined into a single web page that can be displayed in the UI, exported, deployed, broadcasted, etc.
In this sense, LiveCodes is similiar to CodePen, JsBin and JS Fiddle, as opposed to others like Codesandbox and Stackblitz which offer a file system and a whole virtual nodejs environment to support it.

This is an early proposal to simulate multi-file support. This is mainly inspired by the official Svelte and Vue playgrounds and an attempt to implement that for the official Ripple playground which is powered by LiveCodes.

Overall plan:

• There is no plan to implement a full virtual file system or introduce a new build system (e.g. rollup/rolldown)
• The multi-file support is planned using importmaps + data URLs, which are already being heavily used in LiveCodes.
• The 3 basic editors (markup, style & script) will stay the same and will serve as entrypoints. One or more editor can be hidden using the hideTitle property. When the config object has a files property, the markup, style and script properties are ignored.
• Additional files can be added. The file extension determines file type and used compiler.
• File names can have slashes to simulate directory structure (e.g. components/MyComponent.ripple)
• Additional files have to be imported (or added as scripts or stylesheets) from the main entry file (specified using mainFile property and defaults to index.html)
• Script files (e.g. js, ts, jsx, tsx, vue, svelte, ripple, etc) are compiled to js and made available using importmaps.
• Style files (e.g. css, scss, style, etc) are compiled to css and inlined in the css code that imports them.
• Additional markup files (apart from the mainFile) are not allowed (maybe we can later implement imports for the supported template engines (e.g. pug, mustache, etc)
• There is no plan for multi-page apps (multiple HTML files that can be linked)
• Importing JSON makes it available as a JS object.
• Non-JS imports are not allowed (e.g. PHP, C++, Python, etc.). Maybe we can later figure out how to do that for each language (not in the scope of this proposal)
• The UI will allow adding files by a button on the side of the editor tabs. Double-click on the file name allows editing it. A delete button on the tab allows deleting the file. If the tabs are wider than the available space they should scroll horizontally.
• A new files property will be added to the configuration object to persist state and to allow it to be set from the SDK. It will have this type Array<{ filename: string; content: string; }>.
• A new allowAddingFiles property enables/disables adding files from the UI. It will be set to false by default.
• The ability to add new files will be supported in embedded playgrounds first, then we can look into adding it to the UI of the standalone playground app.
• There is no plan to support config files like package.json, tsconfig.json, .editorconfig, etc. Many of these settings can already be achieved by LiveCodes configuration object.
• There is no plan to implement a file tree viewer.

Notes:

• In addition to handling compilation, other features that handle files need to be updated (e.g. import, export and deploy)

Comments and suggestions are welcome.

[logaretm]

I think this is great for many documentation websites, running containers with stackblitz and similar solution is just not great for showing a lot of runnable snippets within the page.

I think the scope is fair and the left out stuff are not important for most of my use-cases, like the tree viewer, I don't really need it.

I'm wondering about the implementation details tho, AFAIK, the Vue compiler may need access to a fake fs to be able to resolve files on its own, not sure how you will pipe it with a streamlined system that works for vue and for everything else.

Other than that, I think once shipped this will be a game changer for livecodes!

[hatemhosny]

Thank you @logaretm
I very much value your opinion. You have always said this is necessary and I only recently started thinking of a possible implementation.

The main challenge is architectural. The concept of a fixed number of 3 editors (markup/style/script) has been there from the early beignning and everything assumes that. So having a variable number of files was not expected.

Regarding the Vue compiler, LiveCodes does not use a file system for that. Some time ago, support was added for using 2 components (for the use case of having an actual component with the relevant logic, and another one that can pass props to it, to demonstrate its use). You can see in this example that the compiler preserves the import (import Counter from './Counter.vue';). The other component is then compiled separately, converted to a dataURL and added to the page importmap.

<script type="importmap">
{
  "imports": {
    "vue": "https://cdn.jsdelivr.net/npm/vue@3/dist/vue.runtime.esm-browser.prod.js",
    "./Counter.vue": "data:text/javascript;charset=UTF-8;base64,LyogQW5hbHl6ZWQgYmluZGluZ3M6IHsKICAic3RhcnQiOiAicHJvcHMiLAogICJyZWYiOiAic2V0dXAtY29uc3QiLAogICJwcm9wcyI6ICJzZXR1cC1yZWFjdGl2ZS1jb25zdCIsCiAgImNvdW50IjogInNldHVwLXJlZiIKfSAqLwppbXBvcnQgeyBkZWZpbmVDb21wb25lbnQgYXMgX2RlZmluZUNvbXBvbmVudCB9IGZyb20gJ3Z1ZSc7CmltcG9ydCB7IHRvRGlzcGxheVN0cmluZyBhcyBfdG9EaXNwbGF5U3RyaW5nLCBjcmVhdGVFbGVtZW50Vk5vZGUgYXMgX2NyZWF0ZUVsZW1lbnRWTm9kZSwgRnJhZ21lbnQgYXMgX0ZyYWdtZW50LCBvcGVuQmxvY2sgYXMgX29wZW5CbG9jaywgY3JlYXRlRWxlbWVudEJsb2NrIGFzIF9jcmVhdGVFbGVtZW50QmxvY2sgfSBmcm9tICJ2dWUiOwpjb25zdCBfaG9pc3RlZF8xID0geyBjbGFzczogIm10LTgiIH07CmNvbnN0IF9ob2lzdGVkXzIgPSB7IGNsYXNzOiAibXQtNCBzcGFjZS14LTQiIH07CmNvbnN0IF9ob2lzdGVkXzMgPSB7IGNsYXNzOiAibXQtNCBzcGFjZS14LTQiIH07CmltcG9ydCB7IHJlZiB9IGZyb20gInZ1ZSI7CmNvbnN0IF9fc2ZjX18gPSAvKkBfX1BVUkVfXyovIF9kZWZpbmVDb21wb25lbnQoewogICAgX19uYW1lOiAnQ29tcG9uZW50JywKICAgIHByb3BzOiB7CiAgICAgICAgc3RhcnQ6IHsKICAgICAgICAgICAgdHlwZTogTnVtYmVyLAogICAgICAgICAgICBkZWZhdWx0OiAwLAogICAgICAgIH0sCiAgICB9LAogICAgc2V0dXAoX19wcm9wcykgewogICAgICAgIGNvbnN0IHByb3BzID0gX19wcm9wczsKICAgICAgICBjb25zdCBjb3VudCA9IHJlZihwcm9wcy5zdGFydCk7CiAgICAgICAgcmV0dXJuIChfY3R4LCBfY2FjaGUpID0+IHsKICAgICAgICAgICAgcmV0dXJuIChfb3BlbkJsb2NrKCksIF9jcmVhdGVFbGVtZW50QmxvY2soX0ZyYWdtZW50LCBudWxsLCBbCiAgICAgICAgICAgICAgICBfY3JlYXRlRWxlbWVudFZOb2RlKCJkaXYiLCBfaG9pc3RlZF8xLCBbCiAgICAgICAgICAgICAgICAgICAgX2NyZWF0ZUVsZW1lbnRWTm9kZSgic3BhbiIsIHsKICAgICAgICAgICAgICAgICAgICAgICAgcmVmOiAiY291bnRlciIsCiAgICAgICAgICAgICAgICAgICAgICAgIGNsYXNzOiAidGV4dC0zeGwgZm9udC1ib2xkIgogICAgICAgICAgICAgICAgICAgIH0sIF90b0Rpc3BsYXlTdHJpbmcoY291bnQudmFsdWUpLCA1MTMgLyogVEVYVCwgTkVFRF9QQVRDSCAqLykKICAgICAgICAgICAgICAgIF0pLAogICAgICAgICAgICAgICAgX2NyZWF0ZUVsZW1lbnRWTm9kZSgiZGl2IiwgX2hvaXN0ZWRfMiwgWwogICAgICAgICAgICAgICAgICAgIF9jcmVhdGVFbGVtZW50Vk5vZGUoImJ1dHRvbiIsIHsKICAgICAgICAgICAgICAgICAgICAgICAgdGl0bGU6ICItIiwKICAgICAgICAgICAgICAgICAgICAgICAgb25DbGljazogX2NhY2hlWzBdIHx8IChfY2FjaGVbMF0gPSAoJGV2ZW50KSA9PiAoY291bnQudmFsdWUtLSkpLAogICAgICAgICAgICAgICAgICAgICAgICBjbGFzczogInRleHQtbWQgZm9udC1tZWRpdW0gYmctZ3JheS01MDAgaG92ZXI6YmctZ3JheS02MDAgdHJhbnNpdGlvbiBweS0xIHB4LTQgdGV4dC13aGl0ZSByb3VuZGVkIGRyb3Atc2hhZG93LXhsIgogICAgICAgICAgICAgICAgICAgIH0sICItIiksCiAgICAgICAgICAgICAgICAgICAgX2NyZWF0ZUVsZW1lbnRWTm9kZSgiYnV0dG9uIiwgewogICAgICAgICAgICAgICAgICAgICAgICB0aXRsZTogIisiLAogICAgICAgICAgICAgICAgICAgICAgICBvbkNsaWNrOiBfY2FjaGVbMV0gfHwgKF9jYWNoZVsxXSA9ICgkZXZlbnQpID0+IChjb3VudC52YWx1ZSsrKSksCiAgICAgICAgICAgICAgICAgICAgICAgIGNsYXNzOiAidGV4dC1tZCBmb250LW1lZGl1bSBiZy1ncmF5LTUwMCBob3ZlcjpiZy1ncmF5LTYwMCB0cmFuc2l0aW9uIHB5LTEgcHgtNCB0ZXh0LXdoaXRlIHJvdW5kZWQgZHJvcC1zaGFkb3cteGwiCiAgICAgICAgICAgICAgICAgICAgfSwgIisiKQogICAgICAgICAgICAgICAgXSksCiAgICAgICAgICAgICAgICBfY3JlYXRlRWxlbWVudFZOb2RlKCJkaXYiLCBfaG9pc3RlZF8zLCBbCiAgICAgICAgICAgICAgICAgICAgX2NyZWF0ZUVsZW1lbnRWTm9kZSgiYnV0dG9uIiwgewogICAgICAgICAgICAgICAgICAgICAgICBvbkNsaWNrOiBfY2FjaGVbMl0gfHwgKF9jYWNoZVsyXSA9ICgkZXZlbnQpID0+IChjb3VudC52YWx1ZSA9IHByb3BzLnN0YXJ0KSksCiAgICAgICAgICAgICAgICAgICAgICAgIGNsYXNzOiAidGV4dC1tZCBmb250LW1lZGl1bSBiZy1yZWQtNTAwIGhvdmVyOmJnLXJlZC02MDAgdHJhbnNpdGlvbiBweS0xIHB4LTQgdGV4dC13aGl0ZSByb3VuZGVkIGRyb3Atc2hhZG93LXhsIgogICAgICAgICAgICAgICAgICAgIH0sICJSZXNldCIpCiAgICAgICAgICAgICAgICBdKQogICAgICAgICAgICBdLCA2NCAvKiBTVEFCTEVfRlJBR01FTlQgKi8pKTsKICAgICAgICB9OwogICAgfQp9KTsKCl9fc2ZjX18uX19maWxlID0gIkNvbXBvbmVudC52dWUiOwpleHBvcnQgZGVmYXVsdCBfX3NmY19fOwo=",
    // ...
  }
}
</script>

The same approach has also worked with Svelte (example).

This is the plan for the multi-file implementation. Each file will be compiled using its appropriate compiler and converted to a dataURL. Scripts will be added to the importmap, while styles will be inlined in the CSS file that imports them.

I have started making some progress. I will let you know when I have something to show isA.

Thanks again.