The concept of infinite space can simply be related to the concept of constantly adding 1 on a number line. This being said, if the universe is ever expanding, then the universe could be called infinite based on the simple definition of infinity.
If you keep adding 1 that's finite. Only when you perform the operation indefinitely would it be infinity. In fact there are many different 'types' and approaches to infinity in math.
In reality though physics is not axiom based. You can take the basic formulations of physics as axioms, and 'prove' something in that mathematical framework, but it's not like you can ever prove the axioms. Just fail to disprove them with an over abundance of data and approaches. To the leyperson it's essentially proving it.
Space is expanding. But so is the event horizon of visible light left over by the condensation of hot plasma during the early universe. As it cooled it stopped being ionized and became more diffuse letting light pass unobstructed. This visible boundary of the visible expands at the speed of light. Incidentally the last scattering of nuterinos, which can pass through miles of lead and even stars without interaction, is slightly larger and if we ever develop a proper nutrino telescope we may see some very interesting primordial events. See my above posts for clarity.
So when we say space is expanding it depends on what type you mean. The actual distance between stars and galaxies by spontaneous space injection (space itself growing) or if you mean the farthest you can see due to relativity and the hot opaque early universe.