If you want to open your Visio (.VSD) file using Visio Services in SharePoint 2013 then unfortunately SharePoint will not open .VSD files in browser. If you have Visio 2013 then save those files as .VSDX and SharePoint 2013 will open those files in Browser or you can save your Visio file as Web Drawing (.VDW) and they should open in browser as well.
Also please make sure you have Visio Services Enabled.