At this point, you are running up against two issues, both of which can make collecting the money difficult or impossible.
First, you need to make sure you can still sue him for the unpaid balance. You state that it has been four years since you gave him the money. In most cases, you must sue him within four years of the date you lent him the money or the date he last paid.
This time limit is called the statute of limitations period. It's the maximum time after an event that legal proceedings based on that event must start. I don't know the period for all 50 states, but in most states the period is four years. If you do not commence the legal proceedings against him within that period, you are barred from collecting the money through the courts.
Second, he is no longer living in the same state as you. While the terms of the contract are based on the laws in the state in which you entered into the contract, you will have to enforce any judgment against him in his current state of residency. That means you could obtain a judgment against him in California, but have to get that judgment recognized in another state. That is not impossible at all and is quite common, but it will take some time.
The cost of trying to enforce your rights could ultimately exceed the cost of what you are trying to recover. That is the saddest part about the law. You have a valid claim, but you might not have the knowledge or resources to enforce that claim.
There are some collection law firms that take cases like this on contingency. The firm will sue your ex-boyfriend on your behalf and attempt to recover the debt for a percentage of the total amount owed. You get something back in exchange for not having to handle the collection process.