Centralized configuration is one of the things that differ from distribution to distribution, so the '/etc/sysconfig/network' is not available to all linux distributions (should be in redhat-based however). Gentoo has a similar configuration file in /etc/conf.d/net, while in slackware you would edit "/etc/HOSTNAME". They are not as important differences as they may sound, these are just definition files that are included by some init script from /etc/init.d (or /etc/rc.d) to pass the information to 'hostname' app that Sahir mentioned, so that they are preserved upon reboot.
Monsky's highlight is also important -if I got it right-, the machine should be able to resolve correctly the defined hostname through it's local resolver or certain services might fail or cause delays due to timeouts (noticeably the xfree86/xorg X11). You can edit /etc/hosts to address that.
