Open the Rich Text Editor of your assignment, quiz, page or discussion post. From here you can upload your file like normal or choose an existing file from the list on the right under "Files"
Once you upload or click on the file you want, it will create a maroon link as shown below:
From there, you will click on the maroon link (1) and then click on the link symbol (2):
Check the box that says "Auto-Open the Inline Preview for this Link" and then click on "Update Link"
This allows your students to automatically see a preview of whatever file (PDF, docx, doc, xlsx, etc.) you attach to your page/assignment/quiz without having to download it!