Running VDX7 headless

[omersiar]

Thank you for this beauty.

There is no discussion portal for this repo so I opened this issue to discuss to adding a headless option to VDX7 which does not require desktop environment (X11).

Simply I already have built a DX7 hardware emulation using minidexed, I wanted to have a dual boot setup for the raspberry pi adding VDX7 to a headless standalone device.

I am planning to have a wrapper for VDX7 similar to MiniDexed.

Do you think that VDX7 can offer a headless option in the future?

[chiaccona]

I think it's possible, but in order to control the DX7 you would have to send it sysex messages in order to set parameters and load/edit voices. Is that feasible for your application?

[omersiar]

Thanks for the reply.
I was discovering how to implement a hardware UI for the VDX7, I see HD44780 emulation is already there and my raspberry pi has the actual LCD attached, what I am going to do is to see if the Sysex commands are going to be reflected on LCD.

I am already working on (really dirty/hacky) HEADLESS fork, I can compile without X11 dependencies but don't handle toSyth or toGUI events (message.h) at the moment which may result in segmentation fault.

My C skills are not up to creating single binary that works both headless and with X11. What do you think, should I go with compile time headless setup or not bother at all?

[chiaccona]

Sounds like an interesting project.

I just pushed a simple "headless" fork to show a possible approach. This still compiles with X11 but doesn't run a GUI under the command line option "-H". If you wanted to compile without X11 you could just cut out the "DX7GUI" class from Gui.h and main.cc and just leave the new DX7Headless class (and change the Makefile to leave out the .cc files and X11 compile flags, of course). The DX7Headless class has a run() function that runs in a separate thread from the synth, communicating through the toSynth and toGui lock-free queues. Right now it doesn't actually react to message at all, it simply just blocks waiting for a getchar(), and terminates the program when you hit return. So the synth work fine from MIDI, but of course you can't push buttons or see the display.

Now the question is how to handle a hardware HD44780. You won't need my HD44780 emulator, since you'll have a real one - you just need to send the appropriate message bytes directly to your hardware. Look at the processMessages() function in Gui.cc line 277. There is a loop that reads the toGui queue and switches on the messages received. The only messages you need to respond to are lcd_inst and lcd_data, which are the instruction and data bytes to write to the LCD hardware. You will also need to set the three flags pins appropriately on the hardware - read/write, enable, and data/instruction.

Put a loop in the run() function that calls processMessages() periodically - the emulator checks the queue every 40 milliseconds for a 25 fps "frame rate". Note that you do want to sleep for some period time - if you just put processMessages in the run loop without a sleep timer, you will be "spin-waiting" and waste way too much CPU. In C++ std::chrono is the best way to set up a timer.

Note that the DX7 has 2 8-segment LEDs which normally display the current patch number. Since the LCD also shows the patch number, you probably don't need the LEDs, though they do flash and show a decimal point when a voice is being edited.

Note also that there are 2 copies of the HD44780 emulator, one in the synth thread, and one in the GUI. This is just an ugly hack needed to implement the LV2 plugin, because synth and GUI need to maintain separate state in LV2. Similarly, the Message queue interfaces are a little convoluted because LV2 requires a different thread communication method. You can ignore all that messiness, or rip it all out if you want.

Let me know how it goes and if you need any help.

[omersiar]

Thanks,

Headless build now works on Raspberry Pi 3 (ARM cortex-a53, aarch64), 64-bit compiler seems imply "mfpu=neon-vfpv4"
But the issue is VDX7 only output noise (pitch does not seem to change between notes)

After changing raspberry pi audio device to I2S DAC, it is now working perfectly.

[omersiar]

Oh my. this sounds perfect, like coming straight out from professional equipment.

• Volume CC works
• CC program change works

Current issues:

• Mod wheel, Pitch bend does not work
• Pan does not work
• Sustain CC sets but can not unset (maybe due to my midi keyboard)
• All notes off CC getting registered on console but it seems does not have an effect (maybe due to my midi keyboard)

[chiaccona]

That's great!

• Mod wheel and pitchbend are off by default in the DX7. There is an "example.ram" file that has them turned on, which you can test using the command line option "-s example.ram" (include the path to the file). Or copy the example.ram to "~/.config/vdx7/vdx7.ram". You can also turn them on with the GUI version of vdx7, which will save to the vdx7.ram file. Or, harder but maybe what you want to do eventually, is to send sysex messages in the DX7's "Parameter Change" format to set the appropriate parameters. You can find the sysex format online.

• Pan won't work because the DX7 is a mono synth, so there is just one Jack output. You could modify JackDriver.cc and Synth.cc to add another output, copying the mono synth output to both channels, and applying a pan control. Or chain a separate mixer to the jack output.

• Check if sustain is maybe toggling instead of on-off. You can use "jack_midi_dump" to see what is coming from your midi keyboard.

• All notes off is a well known bug in the DX7, this was in the early days of MIDI. Check the README.md file for details.

[omersiar]

Thank you!

• Mod wheel and pitch bend is now working
• Did not know DX7 was mono, never seen one in real life, only through emulation ;)
• Sustain is now working (this was due to midi keyboard configuration)
• All notes off working after switching to example.ram,

One issue:

I am sending Voice SYSEX through https://patches.fm/,
even confirmed that the SYSEX bytes arrives at alsaseq device but it does not change the active voice.

On a second thought, actually there is a write-protection right, so let me try this with an actual X11 setup.

But even if that is the case, how the physical owner of an DX7 should use the https://patches.fm/?

https://synthmata.github.io/volca-fm/
I always assumed parameters edited immediately.

[chiaccona]

Memory Protect needs to be "off" (button near the top left), and the synth needs to be in "sys info avail" mode, which is reached by going to function mode (orange button), then pressing the unlabeled "8" button twice, then pressing "yes". They didn't make it easy! And strangely, they didn't put a label on the "8" button indicating that it sets MIDI functions.

I don't know if there is a way to turn on "sys info avail" directly from MIDI...if so it's buried in the sysex documentation. If there is no way to do it, for headless we may need to implement a hack to turn it on by directly flipping a bit in the HD6303 memory . That would be unfortunate, since the bit may not be in the same place in different roms. It's at 0xF8 in the v1.8 rom, which is in the processor's local memory, not the RAM, so it's not preserved between boots. The memory protect flags are at 0x88, bit 7 and 6 for cartridge and internal. Ajxs's disassembly is very valuable.

Or maybe artificially send the series of button presses with toSynth messages?

[omersiar]

Thanks for the explanation!

Even operating manual confusing about it. I was also thinking flopping the bits and even making messaging through an unix socket (also for LCD), then headless wrapper (python) can ease operating it.

Let's see.

[omersiar]

I see lock free queue implementation is really clever, simple and effective. That was a really good learning. Since multiple consumers are not possible with this approach, I abandoned the unix socket idea, VDX7 beautiful as it is.

I found an alternative ROM which enables SYSEX messages on all MIDI channels and does not require memory protection to be turned off which is perfect for my use case.

https://faq.yamaha.com/usa/s/article/U0008191

[chiaccona]

Nice catch with the SER7 ROM. I've tried it but didn't realize it handled MIDI much better. It's the last DX7 firmware update and has some nice features like display of tuning, and individual function data (pitch bend, mod wheel, etc.) for each voice. It does take more tedious programming of the functions, though.

I pushed another headless example you might be interested in. It reads button presses and data slider from stdin and sends to the synth. You could write button presses from python directly or through a named pipe. Or even a socket, it should be realtime safe because the headless thread acts like a GUI in non-realtime and communicates with the real-time thread through the lock free queue.

[omersiar]

Thanks for the STDIN support, will definitely help.

[probonopd]

Interesting. I'd like to build one for testing how the original firmware behaves in comparison to MiniDexed.
Can this be run without JACK, just using ALSA? (JACK is way too complicated to set up for my taste.)

(Semi-unrelated note to future self: On Debian on WSL, sudo apt-get -y install jackd ; jackd -d dummy & will at least get the GUI launched for experimentation; albeit without sound (since WSL has no access to ALSA, jackd doesn't have it either).

[chiaccona]

It would take a major rewrite of the audio driver to port to raw ALSA (and lose all the real-time/low-latency advantages that JACK is designed for). JACK is pretty much the standard for synths on Linux.

Installing on a modern distro is pretty easy, most will set up real-time priority automatically, and qjackctl is pretty intuitive. Also, if you create a ~/.jackdrc file it will run automatically when you start VDX7. Mine currently reads
/usr/bin/jackd -P95 -dalsa -r48000 -p512 -n2 -Xseq -Phw:DAC
where hw:DAC is my USB sound device (that's the only thing you typically need to change). VDX7 autoconnects audio and MIDI ports so the messy connection business in Jack can be avoided.

Sounds like you may be running on Windows? Too bad WSL doesn't have sound (typical Microsoft!). You could maybe try compiling on Cygwin and using Jack for Windows.