Can I Purchase items from Unregistered Firm?

Yes! You can but you have to give reverse tax if your firm is GST registered.