The Church means an assembly of men united in prayer, not a building.

The Catholic Answers

The word Church has a twofold sense. Its proper meaning is a union or assembly of men united not only in prayer, but also in a definite creed, worship, and obedience. In that sense I speak of the Catholic Church. Or again, it can refer to a building erected for purposes of worship by members of the Catholic Church, and in that sense I speak of a Catholic Church.

Radio Replies Volume 1 by Rev. Dr. Leslie Rumble MSC and Rev. Charles Mortimer Carty