The Fitch Group has recently released its U.S. RMBS Servicer Handbook for the most recent quarter. The handbook shows that many banks offering mortgages have halved the number of staff in their mortgage servicing departments during the last two years. The decision to downsize these departments is primarily the result of these lenders seeing their portfolios shrink.
Another reason relates to the number of foreclosures. According to CoreLogic's National Foreclosure Report released in July 2016, there was a significant decrease in the number of active and completed foreclosures from 2015. With fewer foreclosures to manage, banks no longer needed as many staff.
On average, banks reduced their numbers of full-time employees who worked primarily on mortgages, from 8,000 to around 4,000. This reduction did not occur at once but has been gradual, since 2014.
Non-bank lenders have continued to grow. These lenders typically work closer with borrowers and need to interact with their customers more often. Non-banks have not increased or decreased their mortgage departments over the last two years - they have about 2,000 employees who handle these loans.
Bank employees typically manage double the number of mortgages as non-bank employees. Fitch reports that this is unlikely to change.