A new class of bit flipping algorithms for low-density parity-check codes over the binary symmetric channel is proposed. The algorithms employ multiple bits at a variable node to represent its reliability, and multiple bits a check node to capture the sequence of its syndrome values. The check node update function thus requires a simple bit-shift operation, while the variable node updates require a nonlinear Boolean function. This class of multi-bit flipping (MBF) algorithms is enhanced by the probabilistic gradient descent (PGD) algorithm. The gradient descent algorithm minimizes the variable node energy function which, in addition to the classical term which quantifies the discrepancy between the variable estimate and channel value, also involves an additive term defined as a weighted sum of neighboring check node states. Only the variable nodes with the maximal value of energy are eligible for updating, but the updates are not done by default but probabilistically. The resulting probabilistic gradient descent multi-bit flipping PGD-MBF algorithm combined with rewinding improves the codeword probability of error while keeping the complexity lower than that of the state-of-the-art algorithms of comparable throughput.