Can the network difficulty ever come back down?
yes, it depends on the hashpower connected, if it goes down, the diff will go down.
it’s not a good sign for a network to slow it’s hashpower
I used to make so much litecoin before the rise of difficulty
Sure but it was worth less in dollars or euros or goats or whatever. You should be making the same amount now, just less litecoin (which is sad I agree)