No, you should not apply UK VAT. As Norway is not part of the EU, the sale of merchandise from the UK to Norway qualifies as an export sale. Consequently, the sales price is zero-rated for UK VAT.
In principle, it is only crucial that the merchandise actually leaves the EU, not so much that the purchaser is a VAT registered business in Norway. This means that all evidence of the shipment to Norway will be helpful in supporting the zero-rate.
Normally, exporting businesses need to collect customs and haulage documents, which are usual in international shipments. In your case, the shipment may be relatively small and this may cause the documentation to be limited. The exact UK formalities that you should comply with - correct box in the VAT return, wording of the sales invoice, documents to be kept etc. - would be best checked with HM Revenue and Customs, e.g. on their website (see http://customs.hmrc.gov.uk and use "export sale" for the search).