“Setting out” connotes intention and strong will to do something, which was the preventing of good for he poor and needy people. They are similar to the person who was described previously:

مَّنَّاعٍ لِّلْخَيْرِ

(A preventer of good,) (Al-Qalam: 12)

(  وَهُمْ يَتَخَافَتُونَ… ) (…while lowering their voices): Because they decided to do something evil and did not want to be seen, they lowered their voices. This is an adverbial phrase, describing them when they were walking in the darkness so as not to be heard or noticed. None of them had the intention of backing down; rather, all the verbs relate to their story, indicate their determination and insistence. So, they agreed secretly to cut the fruits in order not to give the poor people. Not only that; rather, they even decided to prevent them from entering the garden: