One idea that jumps to mind is putting the running frameworks in an LXC container (or similar) and monitoring the memory usage of that. Not sure how accurate that is, but it's one avenue. It also still might not be fair because some runtimes can let their heap get pretty big despite not actually needing the memory. You'd want a way to differentiate those from the ones with big heaps that would choke with some memory pressure.