The excavator-magnet combo pulled out five metallic file cabinets through the windows on the fourth floor of the five-story building.
Workers then began tearing down the exterior walls of the steel-reinforced concrete structure to locate other filing cabinets.
The city hall, which suffered the most damage to its fourth floor during the series of earthquakes that struck Kumamoto Prefecture in mid-April, has been off limits as it is in danger of collapsing.
More than 50 file cabinets had remained inside the building after the quakes, according to a city government official.
The government aims to collect all the files by the end of this month and then start demolition work in September.
You can watch a Kobelco machine shred a building on another construction site.