when i want to learn it they tell me that i cant attend the lessons because i cant speak swedish

What courses exactly? What’s your knowledge level? Because you won’t be able to join SAS (svenska som andraspråk) classes until you have a good basic comprehension from for example SFI (svenska för invandrare). Depending on your age, your town should have Komvux (adult education) classes for both SFI and SAS.