To dynamically include other files in your page, your server needs to support some kind of scripting language. This can be PHP or SSI, the most commonly-used ones. See Wikipedia for more info on
Server Side Includes.
It's possible to do client-side includes (plain HTML), but I don't recommend this route.
Note that it's possible to make your webserver interpret SSI commands on HTML pages - just make a change to your .htaccess file. This requires a Unix-based server, such as Linux.