The solution of a linear system of equations constitutes an important part in the field of linear algebra that is widely used in industries like aerospace, aeronautics, solid mechanics, fluid dynamics, oil research and numerous others. A direct method for solving these equations is Gaussian Elimination, which consists of forward elimination and back substitution. We have tailored this method to take advantage of the massive parallelism offered by NVIDIA GPU architectures. Thorough evaluations have been performed for variants of our implementation that exploit different memory features on an NVIDIA Tesla C1060 GPU. Compared to a serial implementation on an Intel Core I7, the execution time for forward elimination on the GPU is reduced by a factor of 183X when using both global and shared memory systems, and by a factor of 185X when using only global memory.