Therefore, if the contact information for a particular document happens to be a postal address, then by all means, use the address element.
Some may argue that the use of "may be used" (vs. "should", or "must") opens the door to other possible uses. However, one only has to look at the use of "may" in other places within the W3C spec to realize that this isn't the case.
"Unlike A, it may only appear in the HEAD section of a document"
Even though they use "may", they really mean "should" or "must".
Instead, an hCard or adr microformat would likely be the best way to mark up your address. The examples page even includes some proper examples of using the
ADDRESS element in conjunction with the hCard specification.