Frame challenge point 1: why would someone put effort into developing a formidable strategic advantage and then keep it hidden?

To quote Dr. Strangelove:

The whole point of the doomsday machine is lost if you keep it a secret! Why didn't you tell the world, eh?

Frame challenge point 2: it is against common sense and military doctrine to congregate a large amount of ships into a cramped space, for the obvious reason that they become sitting ducks even for a simple submarine and its torpedoes.

When nukes were tested against naval targets, it was made clear that the concentration of the test targets was way higher than military doctrine suggested.

Frame challenge point 3: considering the amount of troops involved into the cleaning up you suggest, it would be too costly to silence all of them, either with money or bullets.

That said, no, it is not thinkable that the nation would manage to hide such a secret for so long. Again, you don't want to keep it hidden, and there are too many people involved to really keep everything undercover. Look at the real history of how the USSR got privileged intelligence on the early stages of the nuclear program of USA.

And don't forget that once the USA starts playing with its own nuclear toys they will become able to recognize the telltale signs of a nuclear explosion and quickly make 2+2 with all those strange events happened earlier in the South Pacific.

There are several problems with your idea:

No nation would have the money for this.

The Manhattan project took .2% of the US's GDP to develop, and the US had the highest GDP in the world by a factor of 2. It was 5 times richer than Japan, which was the most significant island nation. Assuming various factors were the same, it'd have taken a full 1% of Japan's GDP. We're talking about a nation significantly smaller than Japan or the US, so it'd easily take 10% or more of their GDP.

Lack of additional resources

This island nation would not only need to have easy access to plutonium/uranium to make their bombs, they'd also need state of the art fabrication and research areas, which would require a lot of extra materials to construct. Even "basic" things like steel, coal, or gasoline are hard to come by out in the Pacific Ocean. There's also the matter of human capital. They'd need to be an island of super geniuses. All the best research at the time was happening at a select few colleges in the US and Europe, so they'd either have to rival prestigious colleges like Berkeley/Cambridge with their own out in the pacific or send their best scholars away for several years to train in those colleges.

Independent island nations didn't exist

It's important to remember that the Imperialism was a big deal. As far as I can tell from a brief wikipedia search NZ and Australia were the only countries in Oceania that had gained independence from the Brits prior to WWII. Oceanic island nations like Kiribati, Papua New Guinea or Nauru didn't gain independence until the 60s or 70s. It's possible that there was one or two island nations that somehow escaped imperialism and would have been able to build a bomb without oversight, but if those countries existed, they would have had to been in contact with Europe enough to get the cutting edge breakthroughs in physics and chemistry. You'd need a nation that wasn't colonized but still had deep contacts with the western world, which didn't exist.

The internet didn't exist

We're so used to the speed of knowledge transfer because of the internet. That wasn't the case in the 30's. An island nation is by definition isolated from the world. They would have to physically ship in the newest papers from the US and Germany through the start of the 20th century. It's conceivable there'd be a year or two gap between a discovery in Europe/America and it filtering out to an island nation.

So in conclusion, they'd need a crazy amount of money they don't have, a lack of imperial oversight to conduct their research while still having enough of a relationship to Western powers to understand all the breakthrough scientific advances happening, and they'd need a bunch of geniuses that could do cutting edge work with limited access to outside ideas and materials.

Unless the nation is also hostile to post-war Japan, the Japanese government might request permission to try locate the wreckage of IJN ships to recover the human remains. While a sovereign nation is of course free to refuse, doing so will invite suspicion of something untoward happened. Obviously, it won't be "they have a superweapon back then", but it could make people, local and foreign, try investigating the possibility of war crimes ("maybe they don't want the ships to be located because it will show they're gunning those in the water"). It's one thing for tens of thousands to keep the secret of the superweapon, it's another thing if they got accused of being war criminals.

As much as the nation attempts to take out the invading fleet in one swoop, there's bound to be a rear guard and other groups that either escaped the battle or at least relayed the situation, especially during the second and third attempts of the doomed invasion. They would notice how all communication from the front line ceased without a single straggler reporting their defeat.

And then there's the fallout. Even if the nation stuff their nuke inside a submarine to wreck the invader fleet, there are bound to be detectable remnant in the air around the world that can be measured decades later. Scientists might assume a wrong baseline, but eventually they'd notice the decaying level can only be explained by an earlier nuclear explosion.

A defending nation doesn't get to choose where an aggressor deploys its forces.

There are a couple of (very) fatal flaws in your scenario. (And any cursory study of WW2 naval history will reveal them.)

Firstly as stated in the title the fleets of one nation attacking another aren't sheep, they can't be penned, corralled, congregated or whatever other synonym you want to use. The attacking nation chooses where, how and when it will deploy its fleet. That means your nation has to react to their decisions. It gets to choose when, where or if to launch a nuclear strike but it can't force the enemy to concentrate every ship in its fleet exactly where it wants them to be. That's not how war works.

Secondly? Even if you could guarantee the destruction of every ship and crew member in the invading fleet? The attacking nation would tend to notice the instant its ships stopped communicating with it and then disappeared. After all it's the kind of thing people tend to notice and talk about (a lot). Equally important? Other neutral nations would notice as well.

There would be civilian shipping of all sorts, flying every flag under the sun sailing to and from your nation when the bomb dropped and tens if not hundreds of thousands of foreign nationals living and working in your nation.

Lastly? Spies! Other nations have them. Nuclear weapons production requires large secure facilities and leaves traces that can be detected. Even if its just some random physicist in another part of the world detecting unusual isotope patterns in the air and/or sea water. And as soon as someone gets suspicious? Resources will be poured into finding out what your country is hiding.

Sooner rather later egg heads will be consulted and the senior leadership of other nations will be hearing whispers about (say it softly) atomic weapons.

EDIT: Also look at the Bikini Atoll tests where warships were in fact corralled and then attacked with atomic bombs. Even in that unrealistic scenario some larger ships remained intact. If a radio on any of them could be repaired (and naval ships were set up for that) survivors could and would almost certainly try get a message out before they later died of radiation poisoning.

Frame challenge: this story would work better with nerve gas or biological warfare. These were seen as the 'doomsday weapons' at the beginning of WWII. Britain was afraid that Germany would use their advanced chemistry to poison the whole of London from aircraft. Germany was afraid that we would do the same to them. The Allies did not use biological warfare on the Axis even when the had air superiority. They had fully developed weapons and did not use them for fear of retaliation.

When the Japanese Navy was taking ports in the Far East from the British, you could reasonably guess where they might go next: probably the nearest port to them and the furthest one from you and your supplies. If the British left a port, they could spread anthrax for the invaders. This is not quite accurate because anthrax was only deployable late in WWII, but a lot more feasible than secretly advancing the Manhattan project. This might be seen as similar to laying mines in territory you were forced to abandon: it is a cost-effective way of doing the enemy harm.

You would not want to boast about it. There would be collateral damage to the other inhabitants. Tough on them, but War is Hell. You could deny you did anything: disease broke out when you stopped looking after the territories and their infrastructure. If nerve gases were used, then it was the enemy trying to clear the territory they were surprised by a change of wind, or unwise bombing of a chemical plant.

In this topsy-turvy world, the first use of nukes would probably stop any covert use of poison gas or germ warfare, as the real threat of a revealed weapon is a better deterrent than a covert attack that you could deny.

You want the exact inverse

Your weapon of mass destruction is not just a strategic asset. Strategically blowing up a stronghold or even a whole army in its assembly area is a win, but the Allies and even the Nazis and Soviets knew better than to not capitalize on what war is won by: PROPAGANDA. Every enemy your soldiers kill, every heroic soldier who finds death by the cowardly enemy, every destroyed tank of the enemy, every shot down plane is perfect to bolster the morale of the homefront and throw leaflets and demoralizing messages at the enemy.

A weapon of mass destruction is SO GOOD at demoralizing the enemy and proving your own superiority, it managed to crack a country that a week earlier sent its young and best on suicide attacks that strategically had negative effectiveness!

Let's look at the country that Japan invaded... or rather: the conglomerate of Warlords that feigned to be one country but in fact several: China, which just so happened to have broken up once again.

Manchuria and Korea were Japanese-controlled and a puppet, East Heibei was a Japanese holding. After the Marco Polo Bridge Incident, the KMT (nationalist China) was in a pickle: They were ill-prepared, and Japan was an existential threat. Getting the support of the other Chinese Warlords (like Shanxi and the Communists) meant, they had to lower their own prestige and standing for concessions. And they were driven back, losing Beijing after 3 weeks. If any of the Warlords had a secret nuke in 1937, they surely would have reacted to the Marco Polo Bridge Incident by using it to claim the Mandate of Haven and uniting China: not surrendering and using it to obliterate one of the Japanese armies or a port, loudly proclaiming they had stockpiles upon stockpiles of them and just feigning it would have given the owning Warlord so much leverage, he'd have become the leader of all of China.

And Japan? The Imperial Army needed to win under the old samurai saying of 勝てば官軍 "The winner is King". If you Win, your actions can't be questioned by your superiors. If you lose you are pretty much expected to die by your own hand to shield your superiors from blame. The politicians would have been quick to broker a peace if China showed they could take their armies out in fireballs. The failure of the military would have given the Politicians all they needed to assert control. Propagandizing the effect of the nuke works in favor of peace here!

Hiding is impossible

Now, Trying to hide the use of the nuke is against all military strategy and reason. It should not be done. And it can't be reasonably done, because you'd need to eliminate anyone knowing about the nuke and its use on your side as well as on the enemy side. And we all know, three men can only keep a secret if two of them are dead. In this case: you'd need to kill every soldier that invaded, every soldier that defended where the nuke fell, and then, because of the nature of nukes, remove the very place where the nuke fell from the map. Which is nigh impossible, even if that location is a tiny sandbar in the Pacific and you shovel it to be no longer there. The contaminants in the ground alone would tell a researcher going there exactly what happened, including the ability to calculate the day it blew up down to a rough hour.

You cannot hide the nuclear explosion.

I once worked in a geological institute and I know that big detonations can be detected via seismographs around the world. I know of one example where an exploding factory could be measured many hundreds of kilomoters away.
Seismographs are a well established technology during the time of WW2 (they exist since around 1875).

• Take two seismographs on two locations with precise clocks and you know the exact time of the event and have two possible locations for the center.
• Take three seismographs on three locations with precise clocks and you know exactly where the epicentrum was.
• Detonate an atomic bomb somewhere and many, if not all countries around the world will know where and when it happened.

With that information it is easy to guess who used the nuclear weapon.
You would need a strong earthquake on the same location and at the same time to obscure the detonation event.

Edit: If this is the first nuclear bomb ever then it may be unknown what the big detonation was, but once nuclear weapons are known the mystery gets revealed soon.

Edit 2 as a conclusion from the comments:
Even though it was theoretically possible to locate the explosion via seismographs and Fourier Transformations, at least the people in our history were not able to do that during World War II (the math existed but was forgotten or not yet known how to use it for that use case).
Thus seismographs of that time were able to detect the shockwaves of nuclear explosions but it is unceartain if it could be detected as such and how good the epicentrum could be located at that time.
As the question plays in an alternate history I leave it to the OPs decision if this is possible or not.

Not in the important respects.

It's possible that they might conceal that the weapon was nuclear.

But the really important question is whether they can conceal a very powerful weapon. And they can not. The enemy will know that a large fleet just vanished entirely without a typhoon or other natural catastrophe to explain it.

They couldn't drop the nukes and remain secret. But there's no reason they couldn't plant nuclear devices underwater and detonate them as the ships go over and wipe them out with the shock waves.

It's not like they'd have to stick it on the ships, just get them close enough. And the ocean will absorb the radiation isotopes fairly quickly.

It's been tested in various scenarios, so we know what happens.

History would already be pretty different.

Independently developing nuclear weapons is a huge undertaking. During our world's version of WWII, only the United States (in collaboration with the UK and Canada), Germany, the Soviet Union, and Japan had their own A-bomb projects, with only the US-led Manhattan Project even coming close to completion by the end of the war. You'll note how those four projects account for the five most militarily-powerful and economically-significant countries in the world at the time, and also Canada.

For another country to develop nuclear weapons five to ten years ahead of the US is certainly not scientifically impossible, but it would be completely outside the capabilities of any state that wasn't wealthy enough to be deemed a world power. (A number of smaller states have developed nuclear weapon programs since WWII, but they relied on the advances made during and after WWII.) Espionage, notably, would not have helped much—nobody was really doing secret nuclear research until about 1939 anyway.

This then implies that in your version of history, there is an additional country—and probably a notable landmass—in the South Pacific, and it's a world power in its own right. This would imply significant changes in the course of world history, depending on the history of your country (Is it a former colony of a European state? How did it gain independence? Is it an indigenous state that resisted colonization? What factors led to its resistance and rise to power?), but certainly it changes the circumstances of World War II.

World War II's Pacific Theatre consisted of two factors: On the one hand, Japan was engaged in imperial acquisition of territory, namely West Pacific islands and the Asian mainland. On the other, Japan and the US were locked in a bilateral struggle for influence and control over the Pacific Ocean. If there's an additional world power in the Pacific, that means it's not a two-way conflict, but a three-way one.

An additional true Pacific power would not only impede Japan's direct expansionism, as your question describes, but also interfere with both Japan's and the US's plays for regional influence. Both countries would be actively pursuing policies of either alliance with, or containment of, your country, for years before the war began. This might be enough all on its own to mean that in this world they don't speak of a "World War II" as we know it. Certainly it goes very differently coming out of the war.

You can hide it for 15 ‒ 20 years, yes

Short answer: Yes, with some reservations

Long answer: Can they...

• "congregate the ships": Yes, a suitable choke point can crowd up the enemy ships, especially if they make a feint that builds enemy confidence that they can pass there unopposed

• "drop the nukes": No need, I would instead leave them as remote controlled mines

• "kill any survivors (if any)": gruesome, but yes

• "clean up the wreckage, and then collect any irradiated material": no, not a chance, simply too much mass

The obvious place where this will leak is from the staff involved in creating/deploying the bombs. You can keep it tight, but any conspiracy involving two people or more, is liable to leak.

Still though, it can probably be kept tight enough. The prime example of keeping a nuclear weapons program under wraps is Israel. There have been leaks, yes, but very few.

"But what about the explosion, and the resulting radiation?"

Here is the thing: no one is looking for that, at that point in time. No one knows about nukes in general.

And not even when knowing about nukes, and looking for them, can we be sure to actually notice their use. The Vela incident for instance, even though direct evidence of a nuke was captured on instruments, was more of a mystery than a finger pointing at a culprit. And that required satellites to capture!

So, assuming the nation can enrich uranium to make a Little Boy gun-type bomb ‒ a plutonium implosion bomb is entirely out of the question for an island nation in this time-frame ‒ and make enough of them, say 10:ish, then yes, they can conceivably wipe out the invasion force and hide the truth until the 1960s.

• Hostile witnesses of the explosion are eliminated.

• The fallout will be diluted by the time it reaches the major nuclear players, and no one is expecting it anyway.

• The wrecks will remain.... but finding a wreck when you do not know where to look for it, is hard. And the island nation may also cordon off the entire area. (Plot generator!!)

• Since Japan is defeated by the Allies, records of this failed invasion might get lost. There will be much wondering: "What happened to the ships?! We know they should exist, and now they are just... gone." (Plot generator!!!)

So, without too much artistic license: yes, this plot works.

In practice it’d be really hard to hide: even if you keep witnesses away, the “weird” stuff (acute radiation symptoms, long-term cancers, unexplained fallout patterns, and a blast that doesn’t match conventional ordnance) leaves fingerprints. WWII also had tons of neutral observers, intelligence networks, and postwar investigators—so you’re not just hiding the explosion, you’re hiding the medical and environmental aftermath for years.

The only feasible "island nation in the south pacific" would have been Australia, which probably wasn't what you were after.

Australia has its own fissile mineral deposits, and enough land to put a secret testing base without anyone noticing.

There are even rumours of Australia unwittingly playing host to the only nuclear detonation not sponsored by a nation state, though that remains unproven and somewhat controversial. (Bill Bryson wrote about it in one of his tour-narrative books.)

It wouldn't be too hard to discredit reports of one "possible nuke" as being "a large meteorite impact", though repeated tests might be harder to cover up.

Maybe.

The use of nuclear weapons against a city will be hard to hide. That being said, if nobody knows what those "novel high-yield explosives" are, will they be able to tell that we're talking about nukes and not something conventional? That might hold the first years, but nations with their own "Manhattan Project" will not be deceived. Send a spy to take soil samples, and the secret is up.

But at sea?

A conventional sub fires a nuclear torpedo into the center of a naval task force. The battleships at the center are all gone. So are the flimsy destroyers of the screen. Perhaps a pair of cruisers survive, from opposite sides of the formation. Both report a single very big blast, followed by a sort of tsunami. That must have been the secondary explosion of the magazines of the battleship closest to the observer, right? One well-aimed spread from one gutsy sub skipper. "Something fundamentally wrong with their magazines and torpedo protection schemes."

And, as you mentioned in the question, neither of the cruisers survived the war, no way to take samples directly from their charred paint ...

The big challenge I see are radioactive isotopes in the environment, spreading from that time and place but not present in older, more distant samples. People could look at glacier cores, or sediments, and find incontrovertible proof. Compare US attempts at atmospheric sampling to detect a Soviet test. Time and distance might help for a few years after 45 -- isotopes decay and get diluted, so only better instruments will provide proof.

I'll add one more difficulty to the long list of thorny problems that other have identified: dropping the bomb. The US had only one type of plane that could delivery a heavy atomic bomb, the Boeing B-29 Superfortress, and they had this plane only because they had been building a series of increasing large bombers for use in a war. The only alternative was the Avro Lancaster, which would have made it a shared mission. The Lancaster was also the latest in lengthy evolution.

So our small island nation also has to hide the existence of a bomber development program...

One issue that is not thought about much is that the idea of atomic energy was already widespread far earlier than most of us realize - H.G. Wells wrote about an Atomic Bomb in his novel "The World Set Free" published in 1914

Dropping an Atomic Bomb in The World Set Free. Wells and the illustrator were constrained by their time

The issue isn't that no one knew about atomic energy or the potential, it is no one was able to figure out how to access and extract that energy. Nuclear fission wasn't demonstrated until 1938. Indeed, once the war was on, almost everyone had some sort of nuclear program - "Tube Alloys" in the British Empire, Uranprojekt by National Socialist Germany, a Soviet program was started in 1942 and Japan, despite it's small industrial base, had 2 competing programs - Army's "Ni-Go Project" and the Navy's "F-Go Project". The United States eventually absorbed much of the information from Tube Alloys and successfully completed the making of a working nuclear weapon - largely because they had the resources and willingness to try virtually every possible method of doing so - much of the Manhattan project was based on finding a economical method of separating U-235 from U-238, for example.

This massive building is just one of the multitude of buildings created to test different ways to enrich uranium to create fissile material

So while an unknown power in the Pacific ocean might understand that Atomic energy was possible, the scientific and industrial resources needed to actually make an atomic bomb simply would be out of their reach. Even the British Empire, with all of their resources, simply couldn't set aside enough to successfully create a nuclear weapon during the War.

There is one very out of the box idea which might allow the independent discovery and harnessing of nuclear energy, but it also requires alt physics as well as alt history. Subatomic particles called muons, which are created by cosmic rays impacting the upper atmosphere can "catalyze" nuclear fusion reactions by displacing electrons. Due to their mass, the muons orbiting hydrogen or deuterium nuclei cause the atoms to approach closely enough to overcome the coulomb barrier and have the nuclei fuse. However, muons have such a short lifespan that these reactions are not self sustaining (otherwise the oceans would have boiled away eons ago)

muon catalyzed fusion

However, if there is some sort of handwave which scientists could use to either generate enough muons in a short period of time or overcome the "sticking" problem, then fusion energy can be harnessed in different ways, as a power generation system for energy production or powering ships and vehicles, or to create a catalytic bomb.

Most of the objections to secretly using such a device still stand - a very distinct seismic event can be recorded, and high levels of radiation could be detected near the event or later as fallout from ground bursts or underwater explosions. However, a nation capable of harnessing fusion energy in the 1930's would overturn virtually every social, political and economic assumption and lead to a much different sequence of events - possibly a WWII with different combinations of Powers in alliance, one nation achieving overwhelming economic dominance that the reasoning behind going to war becomes invalid or a singular nation taking on the world - no Axis or Allies needed.

The 1939 world map would change beyond recognition