Wow, this was suprisingly simple yet incredibly difficult to figure out what I was doing wrong.

Situation:

Server X has two NICs, one in a DMZ VLAN (192.168.1.0/24 on eth0) and one in a private VLAN (192.168.2.0/24 on eth1). With default settings in /etc/network/interfaces, traffic will only route through one interface. No matter what, you won’t be able to ping 192.168.2.0/24.

Solution:

In the interfaces config, add a metric for each interface. This is what it /etc/network/interfaces should look like:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
	metric 0
	address 192.168.1.29
	netmask 255.255.255.0
	gateway 192.168.1.1
	dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet static
        metric 1
	address 192.168.2.31
        netmask 255.255.255.0
	gateway 192.168.2.1
	dns-nameservers 192.168.2.1
	dns-search default.net

Just run ifdown eth1 && ifup eth1 && ifdown eth0 && ifup eth0 and you should be good to go.

Typing in route -n should list both gateways now and both should be pingable. No need to do any fancy routing, port forwarding or using iproute2.