There are circumstances under which you may decide that it's time to get a new lender. Under these circumstances, there are ways you can proceed without costing you an arm and a leg to switch.
Talk With Your Current Lender
First, talk with your lender about your concerns. They may be able to fix the problem and make things right for you. Typically, lenders want to make things work for you. They want you to be happy, as you are the customer and are making them money. Talk with your lender about your concerns. If that does not work, put your complaint in writing to the appropriate person at the company. Usually that will solve whatever issues you're having with them.
If that still does not work, you can ask your lender to sell your mortgage to a different lender. Lenders sell mortgages all the time. In fact, over the course of your mortgage, you may switch lenders several times, keeping the same terms of your loan and only paying your monthly payments to a different company.
Your lender might be able to help you and sell your mortgage themselves. This phenomenon is the best option for you, as they may make a small profit by selling your mortgage, and you win by getting a new lender.
Seek a New Mortgage
Another option for you, if you do not like your lender, is to seek a new lender on your own. Approach finding a new lender just like you did when you had your initial mortgage on your property. Do your homework, find the best deal. You will probably be getting a new mortgage on the same property and simply transferring the loan amount on your old loan to your new one.