Instead of Year filter , use Year parameter to get the user input. This will become the current year.
Now create a calculated field 'Previous Year' using dateadd function.
Do not drag any of those on the filter shelf. Just drag the fields on the rows/columns shelf.
Thank you, works just fine for my requirements.