When I open my website on my browser it shows the Joomla logo on my website tab. Is there a way to change it so that my own custom logo shows up instead?
No, it's a kind of data.
I guess ".ico" means "icon".
I use Texturia. And when I open my joomla folder, then my template folder and then the texuria folder, there it is.
I changed it with IrfanView because I didn't find another program which is able to save "pictures" as ".ico". I'm not sure, but either you save it 16x16px or 32x32px.