There are standards to follow when designing electronics. You theoretically can make whatever you want to design and test it. But if you were to sell it, you will have to go through the process of certification/validation.
These standards change, have expiration dates and become harder to comply as years go by, they become more strict.
For example for EU's Electromagnetic compatibility (EMC) standards, this link, go down to Summary list as pdf/xlsx file and you can download and see all the standards and which have been expired.
For safety standards in EU, Low voltage (LVD) test follow this link, go down and download Summary list as pdf/xls document and see the standards that are still valid or expired.
These standards don't tell you how to design your boards, they define how your boards should behave after you have finished your design. So, if you actually find a way to design your layout they way you describe AND pass these test, well, you might as well become rich and famous because I haven't seen your hookup being used in CE-rated devices, it would save up so much cost on components.
(There are some parts on the standards that DO tell you how to layout/proper distance your components actually, but this is done due to "pollution", aka how much dust you expect your device to gather after years).
Image source: from EN 61010-1:2010
An example for your case would be, if you were to get CE marking for your device, to follow transient tests and low voltage safety standards. Such standards examples include:
- EN 61010-1 (Safety requirements for electrical equipment for
measurement, control, and laboratory use)
- CE marking process for electrical devices
- EN 61326-1 (Electrical equipment for measurement, control and
laboratory use – EMC requirements Part 1: General requirements)
including testing and documentation according to EU directives
For example one test includes to bias with 8kV your AC phase. They also bias with 8kV your neutral as well because you might plug your device on the AC outlet the other way, so considering Neutral 0V will fail the test, as it will short the Main or neutral with your low voltages.
I want to fully understand the reason why it is dangerous to use
non-isolated supply
The isolated power supplies are PHYSICALLY isolated with two coils very close to each other. This means they physically can not pass the main's voltage to low/DC voltage output, whatever happens (except extreme conditions like hitting the PS with a hammer I guess).

Image source
There are many things to take into account when designing AC/DC PS like Main's transients, what happens in case a component fails, lighting strikes etc.
Regarding your questions:
Q1) Some power supplies don't reference DC ground to Earth and it is
floating, in that case maybe it is safe to connect a non-isolated USB
device and they float together?
Whatever passes the EMI/EMC and safety tests. Selecting a proper AC/DC converter is also something you should properly assess. Some AC/DC converters use a safety capacitor with high enough leackage that will make you see 110V if you measure between the output of the AC/DC to the GND. Medical AC/DC power supplies usually solve this issue, they are just a bit more expensive.
Q2) If there was a way to connect device ground to Earth, would it be
safe?
Again, you need to pass the safety tests for that. Its usually safer to connect the (every) return path to Earth because it makes it safer in case AC mains touches your low voltage, but you need to pass the EMC/EMI tests as well, where is you connect the GND/Earth together, makes it harder usually because there is this transient test that biases your mains, neutral and Earth with 2kV spikes so you want your electronics to stay away from Earth (usually)
Q3) Is there any way to make USB offline switcher application safe? Or
safe and also legal?
Usually isolators is one way to do it. Other that passing the tests for CE/safety standards, I suggest following datasheet's recommendations and buying a alredy-existing AC/DC power supply. The AC/DC PS come with verification on some safety standards that you can see on their datasheets.