Ok as far as I understand you want to communicate from Server A => B via SFTP so in general uploading some files right? I would do the following (not tested):
Generate a new SSH-Key on Server A:
ssh-keygen -t ed25519
Upload the public key to Server B:
You could either use
ssh-copy-id username@server_b
or
cat ~/.ssh/id_rsa.pub | ssh username@server_b "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys
Server A is now able to login to Server B via it’s private key