Sometimes we want to delete task that we have added to TFS, But there is a no way to delete using a UI. so, you have to use command promt

cd C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE or
cd C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE

Type the following command:

witadmin.exe destroywi /Collection:<TFS url> /id:<Task id>

where
TFS url = Team Project collection url
Task id = ID that is generated by TFS