If $0$ rectangular tables are used, these seat $8 \times 0 = 0$ people. Therefore $36$ people remain to be seated at circular tables. $36$ is not a multiple of $5$, so some tables have empty places.
If $1$ rectangular tables are used, these seat $8 \times 1 = 8$ people. Therefore $28$ people remain to be seated at circular tables. $28$ is not a multiple of $5$, so some tables have empty places.
If $2$ rectangular tables are used, these seat $8 \times 2 = 16$ people. Therefore $20$ people remain to be seated at circular tables. $20 = 5 \times 4$, so $2$ rectangular tables and $4$ circular tables can be used.
If $3$ rectangular tables are used, these seat $8 \times 3 = 24$ people. Therefore $12$ people remain to be seated at circular tables. $12$ is not a multiple of $5$, so some tables have empty places.
If $4$ rectangular tables are used, these seat $8 \times 4 = 32$ people. Therefore $4$ people remain to be seated at circular tables. $4$ is not a multiple of $5$, so some tables have empty places.
If $5$ or more rectangular tables are used, these seat at least $8 \times 5 = 40$ people, so there will be at least $4$ empty places.
Therefore the only possibility is using $2$ rectangular tables and $4$ circular tables.