Yes, they know your IP address and a lot more without you having to register or enter any information.
Your IP address is sent to the webserver with every page and file request that you make on the Web. That’s how the Web server knows where to send the web page and images that your browser asks for. From the IP address, the owners of the server can derive the general location (country, city and possibly neighborhood) that you are in. The server can also determine the brand of Web browser you are using, what helper applications are available, your screen size, whether you have Flash, Javascript and Cookies enabled and what language your browser uses. If you have visited the site before, it is possible that the site has written cookies to your hard drive, which it can retrieve to look up past-visit information.
The website ip-check.info will show you a list of what can be learned from your browser.
Every web server has a log of the IP addresses that have visited, and what specific pages and files were requested. If the website uses web beacons and /or use cookies they can track additional information about where you go on their site.
When you do register on a website, the site will almost certainly record your IP address, the time, your web browser software version, and possibly the website that referred you there, along with the registration information.
This information is used in several ways – one obvious one is to prevent one person registering multiple times on a site under phony names. Another is to detect banned forum members from re-registering on the forum with new names. Finally, if there is any illegal activity, the server administrators can turn the IP address information over to law enforcement.
In general, I recommend that you run Firefox as your brower, with pop ups blocked and NoScript installed so you can control what Javascripts are allowed to run. When you are visiting websites, you have the option of toggling on Private Browsing. This does NOT make you anonymous, but it will erase the history, cookies or other traces that the webserver may leave behind on your machine.
To browse on the web anonymously by masking your IP address, you would need to look into the use of a proxy server or a VPN service. Many websites maintain lists of proxy servers, so they can block viewers from proxies as well.