This was the seventh month of the year until January and February were added in 153 BC. The Latin word for seventh is ‘septem’.
In about 40 AD the Roman emperor Caligula tried to change the name to Germanicus, his father’s name, but without success. I think September sounds better.
The flower for September is the morning glory, and the birthstone is a sapphire.