If you want to only allow few IP to access the FTP service, and disallow all other IP. How??? You can know add all whole world IP on it, right? 🙂
Instead of add the IP that you want to deny, why you not only allow few IP. Here is the trick!!
If you cpanel do have csf firewall, then it will much easier.
(1) Disable the port 21 (assume that you are use default port 21 for FTP service)
(2) Add below link in the allow list
IP= the IP that allow to access
Done!!! Well Done!! 🙂