The effectiveness of argumentation-assisted STEM practices

This study aimed to determine STEM practices assisted by argumentation-based learning, the attitudes and the opinions of the students towards STEM, their perception of problem-solving skills and the argumentation qualifications. In the research, the mixed design has been implemented. The quasi-experimental design with pre-test and post-test control group has been conducted in the quantitative dimension. The research group consists of 55 students—studying at a state school affiliated to Mus Provincial Directorate of National Education. In the study, the qualitative data were collected using ‘STEM attitude scale’ and ‘problem solving skill perception scale’; and the quantitative data were obtained using ‘STEM interview form’ and ‘Written Argumentation Form’. The results show STEM practices have been observed to be effective in improving the attitudes of the students towards STEM and their problem-solving skills. Also, the opinions of the students are positive and the argumentation qualities of the students are at the first level during the practices.


Introduction
Developments in the field of technology have directly affected the economy as well as the education itself.As a consequence of this effect, the countries have gone through changes in their education systems and have attached importance to the educational approaches that enable them to train skilled people who are equipped with the necessary skills for the business world.One of these educational approaches is the STEM education approach in which the fields of Science, Technology, Engineering and Mathematics are taught by being integrated and linked to the daily life (Yildirim, 2016).STEM education first appeared in the United States in 2001.The basis of this educational approach lies in the decline of the interest of the American students in knowledge of science, mathematics and engineering.The US has emphasised STEM education because this decline will cause US to stay behind in terms of economic and technological areas (Ostler, 2012).At present, STEM education is implemented in different forms in many countries (Banks & Barlex, 2014).Since STEM education improves problem-solving, critical thinking, effective communication and technology literacy, STEM education is practiced in many countries (Akaygun & Aslan-Tutak, 2016).
The business world needs individuals who have problem-solving, critical thinking, communication, teamwork and technical skills (Trilling & Fadel, 2009).This situation means that the technical and professional skills will be needed in the jobs that will exist in the future.Thus, the students should be equipped with STEM content knowledge and professional skills from the early ages (Mathis, Siverling, Glancy & Moore, 2017).One of these professional skills is the skill of argumentation (Schwarz, 2009).The process of argumentation contributes to the development of critical thinking, communication, social behaviour and data collection skills of the students.These skills are necessary in daily life for the individuals to maintain their lives as well as their professional lives.It is necessary to integrate the argumentations into the programmes in order to ensure that these skills are acquired during the argumentation process by ensuring that the students are engaged in the argumentation (Newton, Driver & Osborne, 1999).In this context, it is a question of 'how will the information be acquired to the students effectively and efficiently during STEM practices'.This question puts us at the mercy of how to use the learning-teaching process in a course (Fettahlioglu, 2016).STEM education has been implemented through many strategies, methods and techniques (Capraro, Capraro & Morgan, 2013;Dass, 2015;Han, Capraro & Capraro, 2014).In this regard, one of the practices which has been used especially with STEM education and which has recently been emphasised is the argumentation-based learning approach.
Argumentation-based learning is a learning approach in which the reasons for considering, evaluating, opposing and supporting the data obtained through different sources are recommended (Munneke, Amelsvoort & Andriessen, 2003).From this point of view, the main purpose of argumentation-based learning is to encourage learners to ask questions on a topic, to make explanations on the subject and to encourage scientific inquiry by providing evidence and by supporting their arguments with evidence (Duschl & Osborne, 2002;Keys, Hand, Prain & Collins, 1999).The argumentation-based learning process enables the students to think on a topic and to discuss on it.In other words, the students are provided with an in-depth and multiple way of thinking on a subject (Zohar & Nemet, 2002).
When viewed from this aspect, the argumentation-based learning contributes to the development of many characteristics of students as well as to the STEM education itself.It is observed through the studies that similar to STEM education's effect on skills such as critical thinking, problem-solving, creativity and multiple thinking, the argumentation-based learning has also the same effect (Krajcik & McNeill, 2009;Schiavelli, 2008;Childress, 1996;Cotabish, Dailey, Robinson & Hunghes, 2013;Elliott, Oty, McArthur & Clark, 2001).However, when the related literature has been investigated, there has been no research combining the argumentation-based learning and STEM practices and investigating the effect of these two variables on the attitudes of the students towards STEM, their perceptions of problem-solving skills and their opinions on STEM practices.In this context, the purpose of this present research is to examine the effects of STEM practices supported by argumentation-based learning on the attitudes of the students towards STEM, their perception of problem-solving skills and their views on STEM, and the determination of their argumentation characteristics during their practice.For this purpose, the following research question has been searched for.
'What is the effect of STEM practices assisted by argumentation-based learning on eighth grade students'?
Research questions that guided the study included: 1. Is there a significant difference between the attitudes of the students in the experimental and control groups towards STEM? 2. Is there a significant difference between the perceptions of problem-solving skill of the students in the experimental and control groups?3. What is the level of argumentation qualifications of the students in the experimental group?4. What are the opinions of the students in the experimental group on the STEM?

Research design
This study employed a mixed-method approach and thus, the researchers collected, analysed and drew inferences from both quantitative and qualitative data in the study.It involved the assumption that the use of quantitative and qualitative approaches in combination provides a greater understanding of the research problem than either approach alone (Creswell, 2008).
In this research carried out through the mixed research method, a technology-design course in the eighth grade curriculum of secondary school has been performed according to STEM practices assisted by argument-based learning.A model with a pre-test and post-test control group has been used to determine the effects of these practices.The independent variables in the model are argument-based learning and STEM practices, and the dependent variables are the attitudes of the students towards the STEM, problem-solving skills, their opinions on STEM and argumentation qualifications.The symbolic representation of the study model is given in Table 1.

Study group
The study group comprised of 55 students studying at three different branches of a state school affiliated to Mus Provincial Directorate of Education in the spring semester of 2016-2017 academic year.While two of these classes have constituted the experimental group, the other one has formed the control group.
In the study, 37 students in the experimental group have been included in the qualitative study group to obtain in-depth data on STEM practices assisted by argumentation-based learning.For this reason, the maximum diversity sampling method among purposeful sampling methods has been conducted in the selection of the study group.According to Yildirim and Simsek (2011), it is aimed to create a relatively small group in the maximum diversity sampling and to reflect the diversity of the individuals who may be involved in the party to the question in this sampling to the maximum extent.In this context, the students in the experimental group have been included in the qualitative group.

Data collection instruments
Four different data collection instruments, two quantitative and two qualitative data collection instruments, have been used in this study.

STEM attitude scale
In this study, 'STEM attitude scale (SAS)' adapted to Turkish by Yildirim and Selvi (2015) has been used to determine the changes in the attitudes of the students towards the STEM.SAS is a five-point Likert-type measure arranged as 'Strongly Agree', 'Agree', 'Neutral', 'Disagree' and 'Strongly Disagree'.The original scale was developed by Faber et al. (2013) in 37 items and four sub-dimensions.Yildirim and Selvi (2015) calculated the Cronbach Alpha value as 0.94 for the whole scale.For the subdimensions of the scale, Cronbach Alpha values were found to be 0.88 for mathematics, 0.86 for science, 0.86 for engineering and technology and 0.89 for the skills in the 21st century.The Cronbach Alpha reliability coefficient of the SAS implemented in this study has been found to be 0.89 for the whole scale.

Problem solving skill perception scale
The 'problem solving skill perception scale (PSSPS)' developed by Ekici and Balim (2013) has been conducted so as to be able to determine whether the practices carried out in this research have affected the perceptions of problem-solving skills of the students.PSSPS is a five-point Likert-type measure arranged in the form of 'Strongly Agree', 'Agree', 'Neutral', 'Disagree' and 'Strongly Disagree'.The scale was developed by Ekici and Balim (2013) in 22 items and two sub-dimensions.They calculated the Cronbach Alpha value of the scale as 0.88 for the whole scale.For sub-dimensions of the scale, Cronbach Alpha values were 0.884 for the perception of problem-solving skills and 0.777 for willingness and sense of the perception towards problem-solving skills.The Cronbach Alpha reliability coefficient of the PSSPS used in this study has been calculated to be 0.79 for the whole scale.

Written argumentation forms
The necessary data for the answer to the question, 'What is the level of argumentation qualifications of the students in the experimental group'?, one of the sub-questions of the study, have been collected through written argumentation forms developed by the researcher.In the written argumentation form, a text about a problem situation is presented to the students and the questions are presented in the way that the problem situation will enable the students to reveal the argumentation items.Within the scope of the research, problems related to the design of wind rose, parachute, car and helicopter have been given.A written argumentation form has been prepared for each design.

STEM interview form
To look for the answer to the sub-question, 'What are the opinions of the students in the experimental group on STEM'?, which constitutes the qualitative part of this research, and to identify what the opinions of the students are about the practices, the focus group interviews have been carried out with students.In this context, the STEM interview form (SIF) has been formed by the researchers.For the draft form consisting of five questions, two experts in STEM education have been consulted.In the direction of the opinions of experts, one question item has been removed from the form, and 'SIF' consisting of four questions has been formed.Like individual interviews, focus group interviews are important in the qualitative data collection.The focus group interviews are conducted when individual interviews are inadequate in terms of difficulty and time.The main purpose of the focus group interviews is to obtain in-depth, detailed and multi-dimensional qualitative data about the subject or study done in the groups (Cokluk, Yilmaz & Oguz, 2011).In a focus group interview, a student answering a question will also activate other group students.This will increase the extent and depth of the answers given to the question.According to Yildirim and Simsek (2011), this feature of the focus group interviews is also important for such interviews to form rich datasets.

Data analysis
The data obtained in the study have been analysed in two different ways.The quantitative data have been analysed at the first stage, and the qualitative data have been analysed in the second stage.The SPSS package program has been used in the analysis of the quantitative data.Whether the distribution of the scores is suitable for normal distribution has been tested in order to determine which of the parametric/non-parametric analysis techniques should be used in the analysis of quantitative data.Since the datasets obtained in the study is smaller than 50, the data have been examined through the Shapiro-Wilks method.Shapiro-Wilks is one of the methods used to determine whether the scores obtained are normally distributed (Buyukozturk, 2011).The results of the normality test on SAS and PSSPS used in the study are presented in Table 2.When Table 2 is examined, it can be concluded that each dataset presents a normal distribution (p > 0.05).This means that the parametric tests can be applied to the data.Therefore, one-way analysis of variance (ANOVA) will be applied in the analysis of the data.Before the ANOVA technique has been applied, whether the assumptions of the ANOVA technique are provided have been examined.For this reason, whether the scale is equally spaced, whether the data show normal distribution and whether the variances are equal have been investigated (Buyukozturk, 2011).Since the SAS and PSSPS are equally-spaced scales, the first assumption is provided.For the second hypothesis, it has been determined whether the scores of the dependent variances are normal distributions, and the normal distributions have been found (see Table 2).Lastly, for the third hypothesis involving the equality of the variances, the Levene test has been applied to the SAS and PSSPS scores, and the results are presented in Table 3.When Table 3 is investigated, the SAS p-value is more than 0.05.This indicates that variance homogeneity is provided.Thus, it has been decided to perform one-way ANOVA in the analysis of SAS data, which provides all the necessary conditions for ANOVA.However, the p-value of the PSSPS is less than 0.05.This indicates that the variance does not provide homogeneity.Therefore, Welch and Brown-Forsythe tests have been applied as alternatives to PSSPS for ANOVA (Durmus, Yurtkoru & Cinko, 2103;Field, 2009).
The descriptive analysis method has been used in the analysis of qualitative data obtained through the written form.The reason why the descriptive analysis is used in the scope of the study is to present the interpretation to the reader in an orderly manner (Yildirim & Simsek, 2011).Data have been analysed by two investigators in order to increase the validity and reliability of the analysis of qualitative data.The written argumentation forms have been evaluated in accordance with the argumentation rubric developed by Erduran, Simon and Osborne (2004).The qualifications of these argumentations in this rubric are evaluated in five stages.In the first level, only assertions are placed.At the second level, assertions, data, justifications and supporters are involved.At the third level, there is an assertion, data, justification, supportive and weak denial.At the fourth level, it contains assertions that are designated as denial issue.At the last level, it contains claims with more than one denial (Erduran et al., 2004).
Semi-structured focus group interviews have been conducted by using the SIFs in the experimental group upon the experimental practices.As a result of these interviews, approximately 70 minutes of voice record has been received.The sound records obtained in the interviews are transferred into a written form.Firstly, the data are coded separately by the researchers.As a result of the initial coding process, it has been determined that 35 of the codes determined by the researchers are common and the other 15 codes are not common.In terms of initial coding, the coding reliability has been calculated by using the formula ((Consensus/Consensus + Dissensus) * 100) (Miles & Huberman, 1994).The coding reliability for this study is (35/35 + 15) * 100 = 70%.Later, the researchers and the expert have discussed on 15 codes.As a result of this discussion, seven codes have been added into the study and the remaining eight codes have been opted to be removed.

Practice process
The study has been carried out in three stages.In the first stage, the experimental and control groups have been identified.It has been examined whether the three different eighth class branches in the study are equal to each other.For this purpose, pre-test results of SAS and PSSPS have been investigated.Upon a statistically insignificant difference according to pre-test results, the branches have been randomly divided into control and experimental groups.One of the branches has been assigned as the control group (C), another as the first experimental group (E1) and finally, the third branch as the second experimental group (E2).In E1 and E2 groups, STEM practices assisted by argumentation-based learning have been performed and these practices are identical in the experimental groups.The reason for choosing two experimental groups in this way is to test the consistency of the results of the dependent variable tested for effectiveness.
In the second stage of the practice, while the STEM practices assisted by the experimental group and the argumentation-based learning have been carried out, the current technology-design programme has been applied to the control group students.The practices have lasted for 4 weeks.
In the third stage of the practice, the experimental and control groups have been subjected to posttest practices.Whereas SASs and PSSPSs have been applied as pre-and post-tests in all three groups, the SIF and Written argumentation form have been applied only to the experimental groups.

STEM education practices assisted by argumentation-based learning
STEM practices assisted by argumentation-based learning in the experimental groups have been conducted for 4 weeks, 2 hours per week.The data related to the stages of the practiced programme are seen in Table 4.

Findings related to the first sub-question
The descriptive statistics of the scores obtained by the students from SASs according to the groups and measures are shown in Table 5.When Table 5 is examined, it is seen that the mean scores of the SAS have increased in all the groups upon the practices, but the increase in the experimental groups is more than that of the control group.A one-way ANOVA has been performed to examine the statistical significance of these relevant increases, and the results are presented in Table 6.When Table 6 is examined, it is seen that the SAS post-test scores of the students in the experimental and control groups differ significantly compared to the groups they are in [F(2,52) = 8.089, p < 0.05].A 'Tukey' test has been conducted to determine which groups this difference is in.According to the Tukey test results, there is a significant difference between the two experimental groups and the control group.When the mean scores in Table 5 are examined, it is determined that the relevant differences are in favour of experimental groups.Another finding from Table 6 is that there is no significant difference between the experimental groups.

Findings related to the second sub-question
The descriptive statistics of the scores obtained by students from PSSPSs according to the groups and measures are given in Table 7.When Table 7 is investigated, the mean scores of the PSSPS are very close to each other before the practices.However, it is comprehended that there is a visible increase in the PSSPS scores of the experimental groups upon the practices.Welch and Brown-Forsythe has been applied to understand whether these increases are meaningful, and the results are presented in Table 8.That the p-values in Table 8 are smaller than 0.05 indicate the presence of a difference between the groups.The 'Tamhane's T2' test has been applied to determine between which groups this difference takes place (Durmus et al., 2013;Field, 2009).The results of the Tamhane's T2 test for PSSPS are shown in Table 9.When Table 9 is examined, it is seen that the PSSPS post-test scores of the students in the experimental and control groups differ significantly compared to the groups they are in [F(2,52) = 8.723, p < 0.05].According to the results of Tamhane T2 test, there is no meaningful difference between the experimental groups whereas there is a meaningful difference between the two experimental groups and the control group.

Findings related to the third sub-question
In this stage, the written argumentation forms have been implemented to identify the qualifications of the argumentation items that Grade 8 students have used during STEM practices.Four different written argumentation forms have been used in the research for four different activities.These forms have been applied upon the STEM practices, and a total of 20 written argumentation forms have been obtained.These forms have been analysed by considering the rubric formed by Erduran et al. (2004).Findings regarding the qualifications of the argumentation items in the written argumentation forums filled in upon each application are presented in Table 10.When Table 10 is examined, it is seen that the assertations that the students reveal during the practices mostly take place at the first level.Examples of the answers given in the written argumentation form are as follows: S12: The three-leaf wind rose produces more energy (Level 1).S1: I think the three-leaf wind turbines are more efficient and durable since as the number of leaves decreases, the speed of rotation increases and as the speed of rotation increases, the energy produced increases (Level 2).S4: If the parachute width is large, the egg will stay in the air more and the time of fall will be longer because as the surface area expands, the surface area affected by the air will increase.In this case, the parachute will descend slowly and the egg will not break (Level 2).S8: A medium-sized and serrated wheel goes further because big wheels have difficulty in moving due to their weight (Level 2).When Table 11 is examined, it is understood that assertations by students during the practice mostly take place at the first level.Examples of the answers given in the written argumentation form are as follows: S13: Three-winged wind roses are ideal because the three-winged wind roses are commonly used as far as I am concerned (Level 1).S5: If you use thin and small tires, it will go faster because the engine power applied will be higher, namely when we speed up, the tires turn faster and take more roads owing to the lower weight of the tires (Level 2).S7: Three-winged wind roses need to be used because if the double-winged wind rose is used, the balance problem will arise.For this reason, three-winged wind roses must be used.It will also be balanced because the angle among the three-winged wind roses is 120 degrees.This means that three-winged one should be used (Level 3).

Findings related to the fourth sub-question
The students in the experimental group have been asked what their opinions on the practices are.The answers of the students have been analysed and are presented as the findings in Table 12.When Table 12 is examined, it is seen that groups have positive opinions about STEM education upon the practices.When the opinions of the groups have been examined, they have stated that STEM practices are entertaining, provide meaningful learning and increased interest in the courses.In addition to this, other answers are also popular answers, which, in turn, give concrete learning, enable interdisciplinary relationship and allow group study.The answers of some groups on this question are as follows: G1: The practices we have made in the class have enabled us to understand the subject better and have fun.
G2: Concrete learning takes place because a product emerges as a result of the practices we have made.
G5: It provides the development of our handicrafts.

G3: It has contributed to learning by living, doing and participating in practices in the class.
The students in the experimental group have been asked what subjects they have learned during the practices.The answers given by the students are shown in Table 13.When Table 13 is investigated, it is understood that the groups learn most about motion, frictional force and energy conversions under physics lesson.It is also comprehended that the groups learn support motion and nervous system under the biology lesson and that they learn mostly field measurement, ratio-proportion and length measurement under the subject of maths.
The students in the experimental group have been asked what the negative aspects of the practices are.The answers given by the students are shown in Table 14.When Table 14 is examined, it is understood that the problems experienced during the STEM practices are noise and that the applications are time-consuming and are not suitable for the group study.The answers of some groups on this question are as follows: G1: We do not sometimes understand that we have talked with our friends during their practice.G7: We will not be able to use the materials consistently because the materials we use during the practices are expensive.It has also caused us to stay away because we are afraid that it would break.

G3: We have had a lot of difficulties during the exercises and the difficulties in training during the lessons because the practices, which we have, take much time.
The students in the experimental group have been asked who the scientist is.The answers given by the students about the scientists are shown in Table 15.
Table 15.Data related to the question 'Who is the scientist'?Codes G1 G2 G3 G4 G5 G6 G7 G8 G9 He/she is a woman and a man.
x x x He/she is intelligent and accessible to criticism x x x He/she is the explorer x x He/she is the researcher x x He/she is the person dealing with useful issues x x He/she works in the laboratory x x He/she is the person working for human being and society x He/she is a multiple thinker x He/she is hardworking x x: indicates which code/codes correspond to the answers given in the focus group interviews.
When Table 15 is examined, it is seen that most of the students say that scientists can be both man and woman and that scientists are intelligent and accessible to criticism.In addition, scientists have been defined as people who explore, research, work in the lab and engage in useful issues.The answers of some groups on this question are as follows: G1: People who are discovering something in our society are called scientists.Besides this, scientists are both women and men.G3: Scientist is the person who conducts researches and studies related to science, studies beneficial to the science world and gets famous in this world.G4: Scientist is the person who is intelligent, accessible to criticism, intertwined with technology and conducts useful things for society and people.G6: Scientist is the person who is engaged in the development of humanity, constantly tries to find new things or adds something to existing ones, pursues inventions that will affect the history of humanity.G9: He/she is the person who thinks multi-directionally and spends his/her whole day in the lab.

Discussion and conclusion
In the context of the first sub-question of the study, it has been investigated whether the practices in the experimental and control groups have created a change in the attitudes of the students towards the STEM.SAS scores of the groups have been examined before the practices have been implemented, and there is no significant difference among the scores.However, after the STEM practices assisted by argumentation-based learning, it has been concluded that there is a difference between experimental and control groups in favour of experimental groups.In other words, STEM practices assisted by argumentation-based learning have positively improved the attitudes of the students towards the STEM.This result indicates that the practices, which were conducted during the process, have had a positive effect on improving the attitudes towards the STEM.When the related literature has been examined, no study has been conducted to examine the effect of STEM practices assisted by argumentation-based learning on the attitudes of the students towards the STEM.Therefore, this study is accepted as the first study in which argumentation-based learning and STEM education are used together and the effect of these two independent variables on STEM attitude is examined.Besides, there is no direct study regarding the effect of argumentation-based learning on STEM attitudes.However, there have been many studies on the effect of STEM practices on STEM attitudes (Hudson, English, Dawes, King & Baker, 2012;Ozdogru, 2013;Seong-Hwan, 2013;Song, Shin & Lee, 2010;Sung & Na, 2012;Yamak, Bulut & Dundar, 2014).
In the scope of the second sub-question of the study, it has been researched whether the practices in the experimental and control groups have brought about a change in the problem-solving skills perceptions of the students.Before conducting the practices, all the groups have been evaluated in terms of PSSPS scores, and although there has been no significant difference among the scores at this stage, it has been observed that there is a significant difference between the PSSPS scores of the experimental and control groups in favour of the experimental groups upon the STEM practices assisted by argumentation-based learning.Therefore, it is shown that STEM practices assisted by argumentation-based learning have a positive effect on improving the problem-solving skills perceptions of the students.
In the literature, up to the present, no study has been conducted to examine the effects of STEM practices assisted by argumentation-based learning on problem-solving skills perceptions of the students.Therefore, this study is the first study in which argumentation-based learning and STEM education are implemented together and the effect of these two independent variables on problemsolving skill perception is examined.Kim and Choi (2012) studied the effect of the science-based STEM programme on problem-solving skills.As a result of this study, they reached the conclusion that the STEM programmes have developed the problem-solving skills of the students.When the literature is examined, there are studies indicating that STEM practices have positively improved problem-solving skills (Childress, 1996;Cotabish et al., 2013;Elliott et al., 2001).In addition, in the related literature, there are also studies showing that argumentation-based learning has an effect on problem-solving skills.Kardas (2013) studied the effect of argumentation-based learning in science teaching on problem-solving skill.As a result, it is understood that the argumentation-based learning has positively improved the problem-solving skills.Thus, it is concluded that the argumentation-based learning has a positive effect on problem-solving skills.
For the third sub-question of the study, it has been examined what the argumentation qualifications of the students are.As a result of the research, it has been seen that the argumentation qualifications of the students during the practices are generally at the first, second and third level but it has been observed that none of the qualifications of the students during the practice are at the fourth and fifth levels.This study is expected to contribute to the field because this study was the first study to reveal the argumentation qualifications of the secondary school students during STEM practices.
In the fourth sub-question of the study, the general opinions of the students for STEM practices assisted by argumentation-based learning have been examined.When the results are investigated, it has been seen that the students have expressed that the STEM practices are fun, that the practices provide meaningful and concrete learning, lead to an increase in the interest in the courses and enable interdisciplinary relations.In addition to this, they have also stated that the practices give effective results, especially in critical thinking, creativity and in the discovery of new products.When the literature is reviewed, it is seen that many studies have similar results (Baran, Bilici, Mesutoglu & Ocak, 2016;Choi & Hong, 2013;Elliott et al., 2001;Kim, Ko & Han, 2014).In this context, our study supports the results of similar studies in the literature.
Another result obtained in the fourth sub-question is related to the opinions of the students on the subjects they have learned during STEM practices.It is concluded that students have learned the topics such as motion, frictional force, energy conversions and simple electrical circuits in physics; nervous system and support motion system in biology, transformation geometry, ratio-ratio, gold ratio, weight and length under the mathematics course during practices.Moreover, the result under the fourth sub-question is the opinions of the students about the negativities they have experienced during the implementation of the STEM practices.The students have reported that the practices are time-consuming, that there is noise during the practices, that the practices are not suitable for group study and that the materials used for the practices are expensive.When the literature is investigated, there are similar conclusions about the negative aspects of the practices.Yildirim (2016) asked the students in the seventh grade to tell the negative aspects they encountered during the STEM practices.In the study of Yildirim (2016), the students stated the similar issues.The students expressed that there was noise during the practice and that the practices were time-consuming.Another result obtained under this sub-question is related to the scientists.The students in the experimental group have mentioned that the scientists are intelligent and accessible to criticism without gender.In addition, the scientists are defined as people finding new discoveries, researching and engaging in for a human being.

Limitations of study and recommendations
Within the scope of this study, whereas the dependent variables are argumentation qualifications, the opinions of the students about STEM education, the problem-solving skill perception and the attitude towards the STEM, the independent variables are argumentation-based learning and STEM practices.Thus, this present study is limited to these dependent and independent variables.The research is also limited to the spring semester of the 2016-2017 academic year and the secondary school students.When these limitations are taken into consideration, the studies to be carried out on STEM practices assisted by argumentation-based learning can be conducted with the students studying at different grades.
As a result of the study, it is concluded that the number of studies examining the effects of STEM practices assisted by argumentation-based learning is not as sufficient as expected.Hence, it is possible to conduct the studies in which these variables are used together.In addition, it has been determined that the level of argumentation of the students is level 1.Therefore, experimental studies can be carried out to increase the level of argumentation of the students.
In the scope of the study, it has been comprehended that there are some negativities related to STEM practices (such as noise).The practices can be designed taking these negativities into account in the studies to be conducted on STEM practices.
In this study, it has been determined that STEM practices assisted by argumentation-based learning have a positive effect on the attitudes towards the STEM and the perception of problem-solving skills.The effect of argumentation-based learning and STEM practices on these variables can be examined by conducting similar studies.Furthermore, this study can be contributed to the literature by using other types of mixed research method.

Table 12 . Findings of the question 'What are your opinions about practices'?
which code/codes correspond to the answers given in the focus group interviews.

Table 13 . Data related to the question 'What are the subjects you have learned during the applications'? Subjects Codes G1 G2 G3 G4 G5 G6 G7 G8 G9
which code/codes correspond to the answers given in the focus group interviews.

Table 14 . Data related to the question 'What are the negative aspects of applications'?
which code/codes correspond to the answers given in the focus group interviews.