I will do an optimisation run soon to get it running a bit faster. I want to get vertices to recognise when they are inside a polygon, but are closer than the face to the camera.
My current problem is that my cross product always returns (0,0,0).
Let P1, P2 and P3 be vertices that belong to the face being checked.
This code is supposed to calculate the plane's normal based on the three sets of coordinates.
PV1x = Px2 - Px1
PV1y = Py2 - Py1
PV1z = Pz2 - Pz1
PV2x = Px3 - Px1
PV2y = Py3 - Py1
PV2z = Pz3 - Pz1
-- now find the cross product, which is a line perpendicular to the plane
-- PV1 X PV2 = normal vector!
i = (PV1y * PV2z) - (PV1z * PV2y)
j = (PV1z * PV2x) - (PV1x * PV2z)
k = (PV1x * PV2y) - (PV1y * PV2x)
The values of i, j and k are always 0.