The web UI will only upgrade plugins (if you have your server's permissions enabled to configure that) it will not download a new version of Moodle core code for you. As you have seen, the Upgrading documentation details several options for doing this.
You should be aware that you can't jump straight from Moodle 3.4 to the latest version, you will need to upgrade in two steps with an interim version in between. Check the upgrading notes for your target version to see the minimum supported version to upgrade from. You will also need to ensure that the version of PHP you are running on is supported by both the version you are upgrading from and upgrading to at each step.
Doing this in a test environment first is definitely the right move, especially if you are unfamiliar with Moodle. Make sure you can repeat the process with confidence before you do so on a live system.