Very easy.
Once you arrive into Brisbane (assuming you're flying in via the domestic terminal) you can catch the free and very regular (5-10 minutes) transfer bus from the outside of the domestic terminal. This will drop you off at the international terminal.
At Brisbane, you clear security, and then there is a special queue at passport control that says "Norfolk Island passengers", here they'll just check your ID and boarding pass. Obviously as mentioned above in other posts you can travel to Norfolk Island with just a drivers license (and it's technically even possible without any ID but I haven't seen anyone try that) however a passport is preferred because once you've cleared outbound border controls you can shop at duty free but only if you have used your passport.