Generally with the PCIe one you're running something like OpenCL and with system CPU package you run threads and processes like you normally would.
Technically you could run software directly on the old addin cards since they boot to Linux but you had handle the distribution, running and communication of your software with the host. (you could run any x86_64 binary)