From what little I know, one of the problems is prisons today is the lack of treatment for mental illness. Had an uncle who worked in a prison back east, and he said that was a major problem.
If those prisoners were to get out during a SHTF situation, they would ravage the countryside. Also surprisingly, they would tend to use the prison as their base of operations. For many of them, the prison is home. So they would have a heavily fortified base of operations.
Also with the gangs, would they try to settle scores with one another during the initial event?