Hey guys, I'm not that experienced with projection matrices, but I have an idea for a vertex shader I'd like to experiment with. This is where the community comes in.

I'd like to know if it's possible to remove the pitch component of my view projection matrix, or rather, if it's possible to make a copy of the matrix as if the pitch was set to zero.

Can this be done shader side, or do I have to precompute this and pass it in as a uniform?

What are the steps I would take to achieve this?

Your problem is not well defined.

A projection matrix doesn't have an inherent pitch value.

Every 3D rotation can be constructed by combining a yaw, a pitch and a roll rotation after each other, but the specific yaw, pitch and roll angles depend on the order in which youy apply them.

That is: roll after pitch after yaw (Y * P * R) isn't the same as the same yaw after the same roll after the same pitch (Y * R * P).

I will interprete your question further as: can one construct a pitch rotation such that applying the projection matrix after the pitch rotation results in a rotation that can be constructed by applying yaw and roll rotations only.

This is most definitely possible and can be done at shader level.

For reference, definition of yaw, pitch and roll transformations can be found here:

Yaw, pitch, and roll rotations

planning.cs.uiuc.edu

Let's first consider a 3D rotation matrix as projection matrix.

Let this matrix M1 have the form:

Code:

```
[a1,b1,c1]
[d1,e1,f1]
[g1,h1,i1]
```

We now apply a yaw rotation Y before M1 to obtain rotation matrix M2.

Let Y have the form:

Code:

```
[cos alpha, -sin alpha, 0]
[sin alpha, cos alpha, 0]
[ 0, 0, 1]
```

Let M2 = Y * M1 have the form:

Code:

```
[a2, b2, c2]
[d2, e2, f2]
[g2, h2, i2]
```

We want to choose alpha such that b2 = 0 (and M2 will look more like a pitch rotation).

We have by definition b2 = cos alpha * b1 - sin alpha * e1 + h1 * 0

This gives us the equation b1 * cos alpha = e1 * sin alpha

Dividing both sides by cos alpha gives us a simple linear equation with as a solution:

alpha = arctan ( b1 / e1 )

If e1 = 0, then we have cos alpha = 0, so alpha = pi / 2 is a solution.

We now apply a roll rotation R before M2 to obtain rotation matrix M3.

Let R have the form:

Code:

```
[1, 0, 0]
[0, cos yotta, -sin yotta]
[0, sin yotta, cos yotta]
```

Let M3 = R * M2 have the form:

Code:

```
[a3, b3, c3]
[d3, e3, f3]
[g3, h3, i3]
```

By definition b3 = 1 * b2 + 0 * e2 + 0 * h2 = 0

We want to choose yotta such that h3 = 0 (and M3 will look even more like a pitch rotation).

We have by definition h3 = 0 * b2 + sin yotta * e2 + cos yotta * h2

This gives us the equation e2 * sin yotta + h2 * cos yotta = 0

This has as a solution yotta = arctan ( - h2 / e2 )

If e2 = 0, then yotta = pi / 2 is a solution.

M3 is a rotation matrix with shape:

Code:

```
[a3, 0, c3]
[d3, e3, f3]
[g3, 0, i3]
```

If we apply this rotation to vector U_Y of shape

then M3 * U_Y equals:

Because M3 is a rotation matrix, we know e3 = 1 or e3 = -1.

If e3 = -1, we choose a new value for yotta, equivalent to te old value of yotta + pi, which is also a solution and provides us with e3 = 1.

M3 is now a rotation that doesn't change the y-value, which is by definition a pitch rotation.

Let P = M3.

We have P = R * Y * M1.

Let R', Y' and P' be the inverse rotations of R, Y and P respectively.

Then M1 = Y' * R' * P.

This means M1 * P' = Y' * R' is a pitch rotation applied after your initial projection matrix, resulting in a rotation that can be constructed by applying a yaw and a roll rotation.

So we have our solution.

Programatically, we calculate alpha using atan, then we calculate yotta (using atan on some linear combinations based on alpha) and then we construct and multiple Y' and R', which gives us the desired end result.

In your shader, you don't work with just 3D rotation matrices as projection matrices however.

You work with general linear 3D transformations in projective geometry.

This means we have a 4x4 matrix that is a combination of rotations, scaling, translations and FOV altering transformations (and skewing transformations, but let's assume those aren't present for now).

The exact solution depends on the order in which you consider these transformations to have been performed.

Let M1 have the form:

Code:

```
[a, b, c, d]
[e, f, g, h]
[i, j, k, l]
[m, n, o, p]
```

If you consider translations and FOV altering transformations to be applied after rotations and scaling, then the rotation and scaling part M2 is given by:

Code:

```
[a, b, c, 0]
[e, f, g, 0]
[i, j, k, 0]
[0, 0, 0, p]
```

If we assume uniform scaling in each direction and let D be the cube root of the determinant of M/p, then the rotation part M3 of the matrix is given by:

Code:

```
[a/D, b/D, c/D, 0]
[e/D, f/D, g/D, 0]
[i/D, j/D, k/D, 0]
[ 0, 0, 0, 1]
```

You can then apply the same steps as for the 3x3 matrix case above, using the matrix:

Code:

```
[a/D, b/D, c/D]
[e/D, f/D, g/D]
[i/D, j/D, k/D]
```

Having said all of the above, I'm not sure why you want to do all of this to a projection matrix.

If I take a look at the shader matrix structure described here:

I'm using the vecmath library to help with matrix maths while I'm converting opengl program to utilize glsl better. After writing question I guess I've 3 small questions: Where does the model mat...

stackoverflow.com

and if I assume GameMaker uses the same convention and you are referring to the projection matrix in this context, then I would assume the projection matrix to never have a rotation other than a yaw maybe for shake effects.

Then again, I'm not too familiar with how GameMaker splits its matrices up.

TL;DR

My apologies for the long and probably confusing explanation.

You'll have to give a more specific definition/description of what exactly you want to accomplish.

Feel free to ask any questions you have!