Stevenson, I am an unwitting expert, having suffered chronic and sometimes severe insomnia for about 8 years or so.
I define two types, situational and true insomnia. Situational is being upset, angry, worried, excited (it can even be over a good thing) and can't sleep. IMO taking sleep aids works for situational insomnia which generally lasts only a night or two. If it lasts longer, then the underlying problem needs to be resolved.
True insomnia has no reason for being. Ordinary day, ordinary evening, feeling just normal physically & mentally, no reason the person should not be able to sleep, and BOOM, can't get to sleep, even when he/she was unable to keep eyes open before going to bed. Mild insomnia means getting to sleep after a while. Moderate, get out of bed, eventually fall asleep. Severe, up all night.
As to what to do - you've probably researched the basics, avoid electronics before bed, exercise during day but not near bedtime, avoid heavy meals late. Try to sleep in quiet room, although if you live in urban area, where neighbors rev motorcycles and have outdoor parties, that can be tough.
If you don't fall asleep in about 20 minutes, get out of bed. You will only stress out more tossing & turning. Make some herbal tea (chamomile is soothing), find a comfy couch, read something low impact. A lightweight novel you've read before, or yes, a cookbook. That has helped me fall asleep on couch and at some point I would wake enough to stumble to bed.
Severe insomnia, nothing. Nothing. Nothing works. Do you get "earworms"? Because I sometimes felt I don't mind being up all night if the noise would JUST STOP. One line of a song repeating over and over and over and over. Every time I would start to fall asleep I would be awakened by the same line playing over and over. (The earworms were different each bout. I now, not surprisingly, LOATHE every song that ever kept me up.) Sometimes playing soft music through headphones would drown out earworm, with moderate insomnia but with the really bad bouts, NOTHING. And no one who has not experienced that cannot understand how absolutely maddening it can be.
So - the one useful thing I learned from a sleep specialist (more on that later), is just keep telling yourself you won't die, even though at 3 AM you might feel like it. You'll be tired. You won't die. I am not joking, believe me, anyone who has been up several consecutive nights hearing the same line waking them up over and over knows this is no joke.
OK - medical. I went to a sleep specialist. She tried to use drugs. Cure worse than disease. No one told me a side effect of Lunestra is panic attacks. I thought I was going totally off my onion, in a constant state of panic verging on terror. It wasn't me, it was the drug. I found that out because at the time I was still at Genentech and they get a 10 day holiday break. Lunestra was no longer working well so I figured during the break I'd stop taking it, insomnia would be easier to handle when I did not have to go to work next day. After a week off Lunestra all the panic symptoms disappeared. I looked it up and that's when I learned it was a well known side effect.
Several years later my primary suggested a neurologist. It was unlikely, but possible, the earworms were caused by micro-seizures which can be treated. After various testing, everything came back normal. So not neurological.
Did research on earworms. The lines that get stuck in our heads are generally repetitive and catchy. Articles I read suggested they can sometimes be driven out by singing something that is repetitive but not catchy. Many countries' national anthems fit that. The US does not, however, both Oh Canada and God Save the Queen do. I can't tell you how many millions of Oh Canadas I've sung to myself! It works with mild earworms but as I said when they are really bad nothing works.
Then later on I tried a behaviorist. By this time I had researched insomnia and earworms so thoroughly that she told me I was already doing everything she would suggest.
I have heard marijuana can help, but I work at a place that is under US Drug Enforcement Agency regulations. We can be tested any time and the feds don't recognize state laws allowing marijuana, medical or otherwise, so I can't try it.
Upshot of all this, I have insomnia!
Don't know if any of this screed was helpful. Best. If nothing else you sure as hell have my sympathy.