Aleksandar:
I set high res picture for terrain and i still have same problem
Try increasing the number of polygons of the mesh instead. That is what I assume Catalina meant with "terrain resolution".
Increasing the resolution of the height map image can only make the problem worse.