Yeah, there are loads of movies from that era which really focused on that seedier side. Besides the ones already mentioned my favourite ones centred on NYC would be Mean Streets; Taxi Driver; Basket Case; Midnight Cowboy; The French Connection and almost anything by Paul Morrissey or Abel Ferrara. A lot of blacksploitation movies from that era also have a lot of that atmosphere.
For a similar sleaziness but from a more European perspective, some Fassbinder and very early Almodovar are great. Also, albeit more recently, Gary Oldman's Nil By Mouth has that atmosphere in spades.
Another great one is the Brazillian film, Pixote.