Shopworker Adeniyi Afolabi, who lives nearby, gave the name of the school as the Ohen Nursery and Primary, and said there were 144 pupils in attendance on Wednesday.
Another local, Zion Munachi, also confirmed the name and the number of pupils. Both said not all children were at the school because of sports activities.
Joshua Yang, of the Lagos State Fire Service, said the nursery area of the building had now been cleared.
"There are no persons left in the rubble," he told the TVC News channel.
Building collapses are tragically common in Nigeria, where building regulations are routinely flouted.
In September 2014, 116 people died -- 84 of them South Africans -- when a six-storey guesthouse collapsed at the Lagos church complex of celebrity televangelist TB Joshua.
An inquiry found extra floors had been added without planning permission.
In 2016, at least 60 people were killed when the roof collapsed at a church in Uyo, the capital of Akwa Ibom state, in the south.