On the N.E. side of the island and close to the present  wharf there is a large cave. Its entrance is hardly larger than an ordinary doorway, but [the cave is] so large inside that a hundred persons could occupy it with ease. Here we kept our provisions and other supplies.
While on that second tour, an incident that would prove to be a turning point in coastal otter hunting history also became legendary in Santa Rosa Island lore. Nidever retold the story to a historian in 1878:
About the first of January  Sparks and some of our men saw a brig…and remarked casually that they were perhaps N.W. Indians [Aleuts]. Coasting or trading vessels being frequently seen in the Channel and the N.W. Indians not having visited these parts for some time, we all took it for granted that the craft seemed to be a trading vessel. This appearance of the N.W. Indians would not have surprised us, as we knew they were likely to come at any time, and having talked the matter over long before, we had agreed to fight them at least as long as we could; to this the Portuguese also agreed. Sparks and Black Steward while hunting together before, had been driven up into the island by these Indians and their supplies captured; but we determined to defend ours as long as it could be done.
One morning a few days after sighting the brig, we were hunting off the head of the Santa Rosa [Island]. It was very foggy, and at about 7 o'clock we started an otter and began running it towards the head of the island.
Black steward was about 1/4 mile from shore, I was nearly opposite him and distant about 300 or 400 yds. farther out, while Sparks was between us and a little to the rear. Just as we were rounding the point the Black Steward called out, "Here come the N.W. Indians." Sure enough, just ahead of us coming out of the fog were 5 or 6 canoes pulling with might and main to cut us off from the shore. Each canoe had two Indians and some of them a third. When Black Steward called to us, the foremost canoe was but a few hundred yards away and the other only a short distance in the rear.
The fog had prevented us from discovering them, while our shooting had indicated to them our exact position. At the first alarm we made a straight line for the shore and our men needed no urging to exert themselves. We all made for a small cove or bay just below the point and lined with thick bushes. Black Steward was the first to reach the beach. Jumping out as soon as his boat grounded, he turned and fired on the foremost canoe, but the powder having partly escaped from his gun the ball fell short. A moment later Sparks reached shore and almost at the same time I jumped out on the beach beside him, amidst a shower of buckshot, the Indians having already opened fire. At that moment the first canoe was not over a hundred yards away and the others close behind. Sparks fired at the foremost canoe, wounding one of the Indians, who fell but raised again just in time to receive my shot, which settled him. This was a reception they little expected and they turned back until a safe distance from us, exchanging shots with us in the meanwhile.
Nidever and his companions continued to shoot at the attackers from the cover of the bushes, killing three and wounding up to five. He stated that thirteen canoes made the attack, and that the Indians' muskets had an "incredible" range of up to a mile. The Indians retreated to the brig, now visible through the lifting fog. The three men buried their supplies and canoes in the sand and waited in hiding for any sign of invaders. Their companions on the island had hidden in the hills as they heard the noise of the attack. The group had agreed to avoid the cave so as to not give its location away.
The following morning the Indians returned in their canoes, apparently pretending to hunt along the shore near the cave.
They gradually approached the cave, passed by it, and repassed it as if without any intention of landing; finally they proceeded to a point 300 or 400 yds. below and there stopped to fish in the kelp just opposite it. Loth to lose this chance, we instructed Black Steward and O'Brien to remain and keep a lookout while we crept down to the point to get a shot at them if possible.
We reached the point unseen and were about to fire, when the men at the cave raised the cry that the Indians were landing. We ran back just in time. Just before we reached the cave Black Steward and O'Brien both fired at the two Indians in the first canoe but missed them. Our shots brought down one of them, whereupon [they] turned and put off, firing as they went. They again went off to the brig.
The two days following, the brig lay becalmed, without any further attempt of the Indians to return. On the third day they sailed away and we never saw them again.
Nidever later found that the attacking ship had been an unlicensed British otter hunting vessel. He felt that the incident "was a severe blow to the N.W. Indians who had for several years been the terror of the coast. This was the first reverse they had met with." While Nidever did not tell of any return to Santa Rosa Island to hunt, he continued hunting in the area with his own license and later settled for a while on San Miguel Island. He is best known for his part in finding the lone woman of San Nicolas Island in 1853 and bringing her to the mainland. George Nidever was lucky to see Santa Rosa Island in an almost pristine state. He lived out his long life in Santa Barbara where he died in 1883.