The sample is the set of cases from which the results were gathered. Most research aims to generalize the results from a sample to a larger population, i.e. to form abstract conclusions from specific instances. Research posits the existence of a population domain from which one takes a specific sample in order to infer conclusions about it. This requires the sample to be generalizable.
Generalizable. A sample is generalizable if results from it apply to a larger population. Every study collects data from a sample but conclusions are invariably made about a population, e. g. a study of college students using browsers might conclude how “people” use browsers. The sample is the students who gave the data but the population to whom the conclusions apply is people in general. One argues the generalization from sample to population by showing the sample is:
Big enough. Many subjects generalize better than few.
Representative. Reflects population distribution of gender, age, … e. g. 50% female.
Unbiased. That the sample was chosen from the population randomly.
So if the sample is not big enough, not representative, or biased, then the generalizability of the research results is threatened. For example, a case study based on one person is rarely enough to generalize to all people. Equally a sample of just men does not represent a general population that is 50% women. Finally, a biased sample is not a good basis for a general conclusion, e. g. to conclude that “Everybody likes me” based on my mother, wife and sister is a biased sample. Describe why the sample has implications for the population concerned based on its size, representativeness and lack of bias.
Sample size. For an experiment, the sample size needed depends on the number of treatment value combinations, e.g. a study of how Usefulness (High, Medium, Low) and Ease of Use (High, Medium, Low) affect web usage is a 3 x 3 study with 9 value combinations or cells. So by the rule of thumb that each cell needs 15 subjects, this research needs about 9 x 15 = 135 subjects. More independent variables and/or values would require more subjects, e.g. a study of how Usefulness, Ease of Use and Gender (Male/Female) affect web usage is a 3 x 3 x 2 study that needs 270 subjects! Check the sample is big enough for the design of your experiment.
Procedure. The procedure describes how the data was collected, whether by face-to-face, telephone, self-administered questionnaire or online website. It states what was done in time order and in a step by step fashion. For example, an experiment procedure might be:
Consent. Ask the subjects to agree to take part.
Preliminary questionnaire. Gather demographic and other data.
Training. Subjects practice what will be done.
Treatment. Subjects carry out the experiment.
Feedback questionnaire. Gather subject data on the experiment.
The description often includes copies of any scripts or tools used, given in an Annex or on a web site, for example: “After entering the room subjects read the study instructions (Annex A), then signed the consent form (Annex B)…” Also describe any procedure deviations, e. g. what was done if subjects talked to a friend when asked not to. Describe the actual data gathering procedure, step by step, including any deviations.
Task. The task is what the subjects were asked to do. Its description includes any instruction script given to the subjects. Also describe any training given to reduce initial skill variations. Was there a debrief? The task must represent what is normally done, e.g. if investigating subjects using wikis, providing an extra “Help Sheet” that normal wiki users don’t have doesn’t improve the task, as now the research doesn’t apply to normal wiki users who don’t have your help sheet. Describe the task and any instructions involved, including any scripts given.
Pilot study. A pilot study tries out the procedure and method tools on a few cases to uncover problems. Gathering data sounds simple in practice but often things happen that you didn’t expect, e. g. people may not understand some questions. The pilot study data is only used to modify the research method, so it is normal to report these “results” in the method section, e.g. “While trialing the procedurewe found that the spiders were mostly active at night so the experiment was conducted between 12pm and 3am.”
Report how pilot testing improved the research procedure or tools.
Ethical considerations. Any research done at a university must be approved by an ethical committee before it is carried out. For human subjects, questions include “Was consent given?” and “Were the subjects fully informed?” For animals it may include “What measures were taken to minimize pain?”
Gathering data. Every research method gathers information about a research question presented in terms of constructs based on a theory framework. For each construct, the method should state how information was actually gathered to allow others to repeat the study. It might also explain why each way was used because how information is gathered can affect the results, e.g. one can measure Fear by heart rate, pupil dilation, asking “How afraid are you?” or observing actions like running away, and the results may differ in each case. One has to do this whether the data collected is qualitative or quantitative. Qualitative research methods like action research, case study, grounded theory and ethnography all have guidelines on this. Likewise in quantitative work there are many ways to measure a magnetic field, so discuss why you chose one way over others. Discuss how and why each construct was investigated in the chosen way.
Reliability and Validity are quite different concepts
Good data. Good research is based on good data that has two properties every researcher desires:
Validity. Validity is whether the data gathered actually represents the construct concerned, e.g. to do research on hypnosis requires a way to confirm the subjects really were “hypnotized”, else a critic could argue that they were just cooperating with the experimenter. Good research requires valid data.
Reliability. Reliability addresses the question of whether the data gathered would be the same if gathered again, e.g. measures of blood pressure at different times of the day often vary considerably, depending on stress, food, lighting and many other things. Good research requires reliable data.
Reliability and validity are quite different concepts.
Validity. In research, it is critical that what is measured represents what it is supposed to. For example, does cranial capacity measure intelligence, as our brains are bigger than those of chimpanzees? As male brains are on average bigger than female brains, this would imply that men are smarter! Unfortunately whale brains are six times bigger than ours and the brains of Neanderthal “cave men” were about 10% bigger. So male brains are bigger because men are on average bigger not because they are smarter. To make the point, using foot size to measure intelligence would be a reliable measure but not a valid measure. Validity in research is established in three ways:
Content validity. Is that the construct is generally agreed to be self-evident in the measure, e. g. asking “Did you enjoy the web site?” as a measure of enjoyment. Content validity is used for established constructs defined in the literature review.
Criterion validity. Involves validating a new measure against one already accepted as valid, e. g. a web clicks “interest” measure can be validated by correlating it with purchase data. Criterion validity is used for a new measure of an established construct.
Construct validity. Is when a new construct is deduced from existing data that “hangs together” (convergent validity) and is distinct from other data (discriminant validity), e. g. a construct like “Economic Setting” might lack construct validity if it is not a unitary variable, i.e. one thing. One can also use factor analysis to define the parts of a multi-dimensional data set that vary together.
The easiest way to ensure valid measurement is to use a tool someone else has validated. It is always important to argue why the data gathered represents what it purports to represent.
Reliability. Good data should change little when measured at different times or by different people. Reliable measures are stable and without errors caused by the measuring itself. For example, questions that different people interpret differently are not reliable. Qualitative studies use standard methods, keep detailed journals and analyze consistency to get reliability. Quantitative studies use:
Test-retest reliability.Involves checking how a measurement changes if repeated, e. g. give a test then give it again a day later to see how much change there is using Pearson’s correlation .
Split-half reliability checks if the measure is internally consistent with itself, e. g. compare the first half with the second half of a test, to see if those who score high on the first half also score high on the second half.
Reliability coefficients like Cronbach’s alpha are generally accepted if they are 0.85 or higher, while others like Kendall’s tau must be least 0.4. Again, to ensure a measure is reliable, use one that someone else has tested and found reliable. Whenever you measure anything, it is always important to establish that the data gathered would give similar results if it was gathered again in the same way.
Accuracy. Accuracy is how close a measurement is to its true value based on the sensitivity of the measuring instrument. It is often expressed in terms of the number of decimal points. Accuracy is not a substitute for reliability or validity, e.g. an accurate measure of blood pressure with hi-tech equipment is still unreliable if a subject’s readings vary during the day. It may also be invalid say as a measure of stress if it varies with activity that is not stressful. It is misleading to use more decimal points than the data reliability supports, e.g. “33.333% of the rats responded to the treatment, 33.333% did not, and the other one escaped during the experiment.”
Missing Values. Missing values occur when a data gathering attempt fails. Attempts to gather data can give three outcomes:
1. A successful response.
2. A nil response (NR).
3. A not applicable response (NA).
Where NR and NA are missing values. For example, when asked “Why did you buy your mobile phone?” the subject may give some reason (a successful response), they may walk away as they don’t want to answer (a nil response), or may say “I don’t have a mobile phone” (a not applicable response). It is important to record missing values as they are also data and may also have implications. Report how missing values were recorded in the data.
Unit of Research. The unit of research is the data collection unit, i.e. one data collecting act or “case”. Often the unit of research is the subject but it can be other things, e.g. in an online group voting study, the case could be a vote (choice measure), an individual (satisfaction measure), or a group (agreement measure). The difference shows in the raw data table, where the columns are variables and each row is a case, e.g. if the rows are people then one person is one case but if they are votes then the vote is the case. The case affects the number of measurements (N) of the analysis, e. g. 90 subjects could give 900 votes (N=900), 90 satisfaction ratings (N=90 people), or 18 five person group agreement ratings (N=18 groups), depending on the case. Unless it is obvious, state clearly the unit of research of the study.
Method type. The method type is how one chooses to gather evidence based on previous research and the data available, e.g. researchers trying to decipher Egyptian hieroglyphics could only guess what the symbols meant until archaeologists discovered the Rossetti Stone, a tablet with side-by-side Greek and Egyptian versions of the same text. In contrast, physicists investigating the atom could carry out experiments to discover its structure. In general, research methods are grouped according to whether they discover, describe, correlate or causally link constructs as follows:
Exploratory. Exploratory methods use data to discover initially unknown constructs, e.g. grounded theory, a method used to study new cultures.
Descriptive. Descriptive methods gather data to better define constructs in complex areas where the constructs are unclear, e.g. case studies or political focus groups.
Correlational. Correlational methods show connections between constructs but not what causes what, e.g. surveys or longitudinal studies in psychology.
Experimental. Experimental methods establish cause and effect by manipulating a independent variable to see its effect on a dependent variable, e.g. laboratory studies.
For example, historians can describe history and sometimes correlate events but cannot experiment on what is past. None of the above methods are better or worse, just more or less suited for the research situation. To use exploratory research for a well-established topic may be as misplaced as to do an experiment in a poorly understood area. Choose a research method that suits your topic area.
Realism vs. control. Research methods differ in how realistic the data is and how much control the researcher has over it. In general, control increases in the order exploratory, descriptive, correlational and experimental, while realism decreases in the same order, e. g. case studies are more realistic than experiments but offer less control. Exploratory research is the most widely applicable but experiments are the most demanding. Choose a research method that is as realistic as possible while still being feasible.
Data types. Data can be qualitative or quantitative. The research method chooses the type of data collected, which can be:
Qualitative. The data is semantic, e.g. a subject statement.
Quantitative. The data is numeric, e.g. a response count.
Mixed Method. A combination of both the above.
For example, to investigate web site usage one can gather the number of views to give quantitative data, or interview people who use it to give the qualitative opinions. The properties of good research, like validity, reliability, repeatability and generalizability, apply equally to qualitative and quantitative data. Each has different benefits, so while qualitative studies struggle for precision, numeric studies struggle for meaning. In practice, qualitative and quantitative research are complementary, so consider a mixed-method approach that gathers both qualitative and quantitative data to get the best of both worlds, like the Repository Grid or Delphi method. Interviewing subjects after an experiment can be very helpful to decide what the results mean. Gather both qualitative and quantitative data for the best results.
Method or analysis theory. If the method is uncommon or new, briefly present the theory behind it, e. g. Delphi method, factor analysis theory, interpretive method theory or conjoint analysis theory. Note that the method theory is not the research theory, as the latter is central to your research but the former is just an aspect of it. So keep it brief – readers who are already familiar with factor analysis don’t want to read pages of factor analysis theory copied from a textbook! Research theory needs extended coverage but method theory does not, so refer readers elsewhere for details. Briefly present the method theory if it is uncommon or new.
Research question. The research question is the abstract question that the research aims to answer by concrete evidence and is the core of any research. It comes at the end of the literature review because to put a question one must define the concepts behind it. It comes before the method as one can’t gather evidence for a question one doesn’t know. The literature review makes the case that it is a good question, given what others have done. A good research question should be:
Unitary. This means it can be stated in a single sentence,although one research question can break down into many sub-questions, a., b., c…
Real. “Does fear make one afraid?” isn’t a real question as fear makes one afraid by definition. This is like asking “Is 1 = 1?” A research question puts a choice with more than one possible answer. If the answer is a foregone conclusion, it is not really research. Good research asks risky questions.
Answerable by evidence. e. g. “What affects life?” is a bad research question as everything affects life and one can’t collect data about everything. A more focused research question makesit more likely that the research will finish.
It can take years to define a good research question, even with advisor help. Often it is trial and error, as one question might be too general to answer while another might have been done before, and so on. Don’t be discouraged if it takes a while, as a weak question makes everything that follows weak. On the other hand, a good question will be good research, even if the data doesn’t show what was expected! After all, knowing what is not true is often just as important as knowing what is, e.g. clinical trials of a drug that doesn’t work. A good research question is a single question that puts a real choice that is answerable by evidence.
State a hypothesis. If possible, state the research question as a hypothesis, a statement whose opposite the research results can deny. The null hypothesis is the opposite of an hypothesis, e. g. the question “Do people prefer polite software?” allows the hypothesis “Subjects use polite software more than impolite software“. This is not “proven” but the null hypothesis that “Subjects will use polite and impolite software the same” can be rejected because assuming that polite and impolite responses are the same, one can estimate statistically if the results agree or not. If the results are very unlikely, one rejects the null hypothesisand so accepts the hypothesis. Argue each hypothesis as a statement whose opposite the results can falsify.
Significance. Significance is the degree to which the null hypothesis is unlikely. It measures the level of confidence in the results. Common confidence levels are:
Likely. A finding is likely if the probability that the null hypothesis is true is less than 5%. This is 1 chance in 20, stated as p < 0.05.
Very likely. A finding is very likely if the probability that the null hypothesis is true is less than 1%. This is 1 chance in 100, stated as p < 0.01.
If p < 0.05, the null hypothesis is rejected and the hypothesis is supported, else it is not supported.
Life is uncertain
Where’s the proof? So where then is the proof in science? In the natural sciences there is none! We don’t live in a perfectly ordered world where 100% proof is possible, as quantum theory confirms that all physical events have some randomness built in. Except for artificial symbolic domains like mathematics, life is uncertain. All science is no more than a set of working hypotheses that are probably true, all open to amendment, e.g. to say that objects fall down by the earth’s gravity just means that we haven’t found an exception, and indeed in theory an anti-matter object on earth would “fall up”. Some find this depressing but for me it makes science interesting. A scientist is one who accepts uncertainty and finds it interesting.
The theory framework is the set of concepts that underlie the research question. One of the jobs of the literature review is to select the theory framework you intend to use and describe it. Note that science uses the term “theory” differently from common use, e.g. if I say “My theory is people don’t listen” that is my opinion and others are entitled to have a different opinion. In contrast in science, not all theories are equal as it is about selecting the best from the rest based on data, e.g. to say “Evolution is just a theory” is to confuse what is based on an enormous amount of evidence with an opinion. The difference is that while opinions often lead nowhere, a good theories lead to better questions, e.g. one asks “Why did God give giraffes long necks?” or “Why did giraffes evolve long necks?” depending on the theory base. One question leads to speculation and the other to a search for evidence. Theory is the beginning and end of research, as theories lead to more research questions. The literature review makes the theory behind the research question explicit. So review the literature to pick the theory that underlies the research question.
Theory directs data. Science is not the search for data but for knowledge based on data, as the world has data without end but knowledge is rare. Knowledge comes from the questions asked of data not the data itself. For example, in 1887 Michelson and Morley went to great lengths to find out if light went at the same speed in every direction not because they cared but to test the theory that everything moved in an invisible ether. Searching for data without theory is like a blind man casting about to invariably find nothing, e.g. student questionnaires that ask every question they can think of. When asked what they are looking for, they say everything but the result is a lot of useless questions that lead nowhere. The key to finding is to know what you are looking for, and theory tells you where to look.
Theory is abstract. Scientific theories connect abstract concepts to evidence from the vastness of data. The power of the abstract is that one theory can describe countless specifics, e. g. the law of gravity applies to all matter, from a falling apple to the moon. The power of science is based on the abstractness of theorybut that is also its great weakness, as it can disconnect from reality. Every paper needs the power of theory, as research without theory is like a car without an engine – it doesn’t go far. A theory is abstract concepts connected.
Topic construct. Every scientific theory is built from ideas but a construct is a concept connected to data evidence, that is “constructed” as it were from data. A construct measured as a number is a variable, e. g. the construct Usage as measured by web hits is a variable. In contrast, the construct consciousness is based on personal experience so science can address it but currently has no measure. The topic construct is the construct the research investigates. If there is more than one, combine them into a higher construct, e. g. combine software Satisfaction and Enjoyment into a more abstract Response construct. A construct and its measure are the same thing at different levels, so if your study measures fear by changes in blood pressure, don’t say that fear “causes” changes in blood pressure. The literature review identifies construct(s) relevant to the topic, i.e. concepts and their evidence base.
The literature review must state the topic construct, e. g. in a thesis entitled “Web Site Effects” the topic construct was unclear, as the question “Effects on what?” was ignored. Possible topic constructs included the customer relationship, user site acceptance, web site usage, marketing, communication, etc., any one of which was a study in itself. The research meandered and gave weak results because the literature review didn’t define a topic construct. Check that the literature review defines a topic construct.
Causality. A literature review not only identifies the key constructs but also how they might cause each other. In describing theoretical constructs, it is critical to distinguish what is causing from what is being caused. Quantitative science divides variables into:
Dependent variables. Variables that are caused, i.e. change depending on other variables. So if the topic construct is a variable, it is always the dependent variable, e. g. for research into how ease of use affects web site use, the latter topic construct measured as number of views is the dependent variable.
Independent variables. Variables that cause others to change. Research often look at independent variables that change a dependent variable e.g. web usage is also affected by the value a site offers.
Moderating variables. Variables that alter causal effects, often individual differences, e. g. expertise may moderate how ease of use affects usage, as experts care less about ease of use and more about value.
Note: A construct can be both a cause and an effect, e. g. if ease of use increases web usage that in turn increases web sales, web usage is both dependent and independent. The literature review must distinguish what is causing from what is being caused.
Theory figure. After a literature review picks the theory used in the research, it is often a good idea to provide a figure. This may be an existing theory, a combination of existing theories, a modification of an existing theory, or a new theory of your own making. Since the theory is what directs your research, it is worth showing it in a figure. A good literature review gives a figure of the theory the research uses.
Logical flow is when ideas follow each other in a rational sequence. A literature review with a logical flow makes the reader feel they are on an intellectual journey. The opposite is to just describe one piece of research after another in an apparently random fashion until all are covered. Just reporting a summary list of what others have done, as A did this, B did that, C did the other, etc. presents as a “What I read in the weekend“ report and is an undergraduate style. A literature review should analyze past research not just report it.Analysis means not just describing past research papers but also looking at how they connect.
Organize by key issues. One way to structure a literature review is by issues, so on one issue compare and contrast many authors, then “rinse and repeat” for other issues. Bring out agreements, conflicts, contradictions and gaps in the literature by looking at how past papers connect to each other. For each issue, maybe start with an archetypal advocate then add in those who agree and disagree. A literature review organized by themes is much easier to follow than one that just lists many papers one after the other. Analyze the literature by themes not just as a sequential list of other research.
Start easy. When writing a literature review, it pays to start with what people agree with, including yourself. Don’t start the literature review by outlining your contentious new theory then explaining why everyone else is wrong. Not only does this offend, it is backward thinking. Start with what is agreed and work forward to points of difference. Then at the end progress to a preferred conceptual framework and a question to test it, i.e. demonstrate forward thinking.Do the analysis rather than start with a research question and then try to justify it. If you do this, you may be surprised where it will lead.
Impartiality. Report other research in its own terms, as those who espouse it would. To describe papers that disagree with your approach in a biased way is dishonest. Even if you think another person is wrong, describe their work like a professional reporter who reports even what they disagree with. By all means report one author’s critique of another but don’t take sides early on. Leave your conclusions till last. Only after honestly representing others do you add your opinion, as one doesn’t evaluate without first understanding. Also, the paper may be reviewed by those who know the works you report, even the authors themselves. Describe current research impartially, in its terms not yours, before criticizing it.
Avoid opinion statements. An opinion statement is a statement that is not necessarily true but stated as if it is without justification. Avoid unjustified opinion statements as they are to reviewers like a “red flag” is supposed to be to a bull, i.e. they invite attack. For example, to say that “Everyone lies” can’t be defended as it is not possible to check “everyone”. It must either be stated as an assumption, or moderated to something like “Most people tell lies at some point in their life“, and even then one needs to reference a study that supports this. Many people saying a thing doesn’t make it true, so statements must be justified by evidence. Avoid opinion statements that are red flags to reviewers.
End with the research question. The whole point of the literature review is to end up with a research question that naturally “falls out” of the previous work. In contrast, a bad literature review describes all the previous work then suddenly, for no obvious reason, jumps to some research question based on an unexamined author bias. Such discontinuity suggests a lack of analytic thinking. Creating a logical flow is like stringing pearls into a necklace – one pierces though beautiful ideas and joins them up with the string of logic.
Collegiality. Collegiality in science means recognizing your colleagues. Science is a social activity as well as an intellectual one, done by a community whose culture is the scientific method. This community resolves disagreements not by conflict but by appeals to logic and evidence, i.e. by research. Scientists who disagree about ideas agree to abide by research conclusions. One respects research colleagues by reading and referencing their research. To ignore other’s work or to copy their words without quoting them disrespects the scientific community. Respect the scientific community by reading, referencing and building on other’s work.
The literature review should cover all the important research relevant to the topic. Describing important researchis like describing well–known land marks while describing relevant research is like describing features that apply to your journey. Both are important, as not describing well-known landmarks suggests you don’t know the area and describing what is irrelevant wastes people’s time. Don’t make the common mistake of describing well-known research that isn’t relevant to your topic.
Important research. Every topic has key authors who have had significant impact and the literature review must show you are aware of them. Even if you disagree with their ideas, show that you understand what they have said. To not do so suggests you are ignorant of what is important. Cover all the important ideas that affect the topic, both old and new. The literature review should cover all important work relevant to the topic, both old and new.
Relevant research. Given the research topic and scope, focus the literature review so as to not waste the reader’s time. Readers of research want one topic treated well not a superficial skimming of many topics. A literature review is not an exercise in showing how much you know. If you want to say many things, then write many papers. However interesting a citation is, if it is irrelevant to your topic drop it. Don’t mention it just to accumulate references. A good literature review focuses on papers relevant to the topic, e. g. I once reviewed a paper on online education authentication that spent a lot of time describing well–known online education authors who didn’t mention the authentication issue. Research outside the topic scope doesn’t add value to the literature review. A literature review should only discuss research that is relevant to the topic
Multi-disciplinary. In modern research in one field can and often does have implications for another, e.g. psychology research can relate to research on web sites. If the topic overlaps another field, consider referencing it, as this can be source of synergy and innovation. Is relevant literature from other disciplines mentioned?
Precedence. Precedence refers to the research that immediately precedes and leads to yours. Given the amount of research done, probably something similar has been done earlier. A good strategy for new researchers is to find a successful precedence paper and carefully follow and learn from it. Establishing a precedent also gives others confidence that your work is worthy. Pay especial attention to such work and give credit to those who have preceded you. As Newton said: “If I have seen further than others, it is because I have stood on the shoulders of giants.” Finding a precedence in accepted research is helpful, as you can use the same method, tools or analysis, and perhaps also avoid any criticism of their work. A good literature review describes the research that immediately precedes your work.
Contribution. Given a precedent, clarify how your research adds to it, to show an original contribution. Research is not just about getting results but also adding value. As a knowledge contributor, you may add to theory or results, or both. Be aware that taking a new theory perspective is challenging, as reviewers wedded” to the current theory will object, so expect a rocky road to publication as genuinely new theories can polarize reviewers, i. e. some love it and some hate it. Unfortunately, the current review system favors conformity, as any one of four reviewers hating a paper can block its publication. Yet if no-one ever differed from established theory science would not progress! Describe any theory contribution in the literature review.
The scope of a topic is like a boundary line around a territory that specifies what is inside and what is outside the area. It defines the problem space you are addressing. State not only what the research is about but also what it is NOT about, by discussing what important others have done that is outside your topic, e. g. if the topic is online grocery shopping distinguish this from other forms of online shopping and from e-commerce in general. When the scope is poorly defined authors wander freely in areas of little relevance to their topic and consequently annoy informed readers.
If the scope too broad the research may not adequately cover it, while a too narrow scope will limit the implications. It can be hard to define a “Goldilocks” scope that is “Not too hot, not too cold, but just right”. For example, for a project on the hydro-chemistry of surface water, be clear on the sorts of water it applies to and the sort of chemistry involved. Don’t be surprised if the scope changes as you progress. The scope description is what others use to assess if the research is relevant say to a journal or conference call for papers, so it is very important.
Define the scope early, as defining it later creates ambiguity about goals. The scope affects the limitations of the research, the data used, the theories that apply and the conclusions made, e.g. it can explain why a certain data or subjects are excluded from the research. It also helps to manage expectations.
Key terms. Key terms are the main constructs that the research addresses. Defining the key terms correctly and using them consistently is critical to properly defining the scope of the research. Give any abbreviations used in full the first time they are used, e.g. ISP (Internet Service Provider). Where possible, use accepted definitions rather than giving terms your own meaning to avoid confusion. If necessary, take the time to define terms that you interpret differently, e.g. my paper on software politeness defined it as giving choices rather than being nice.Always define key terms carefully.
Scope is important because it prevents critics from misinterpreting your research. Stating what research is not about immunizes it against battles you don’t want to be involved in. In this case, scope acts like a disclaimer on a product, such as “Not to be used by infants”, e.g. research on the correlates of cancer may disclaim that it implies any causal links. Define the research scope early to clarify the purpose and avoid misinterpretation.
Publication type tells the reader the paper structure to expect, such as:
Theoretical. Has no method section.
Review. A state–of–the–art summary of existing research.
Empirical. Collects physical data with a method and results section.
Proposal. Outlines proposed research with a method but no results.
For a complex paper, it can help to outlinethe main sections to follow at the end of the introduction: “This paper will first … then …” Indicate the publication type in the introduction so readers know what to expect.
Scientific knowledge in general has three sources, namely:
Domain evidence. The new evidence the researcher has gathered.
Logical analysis. Combining concepts to deduce new knowledge.
Past research. Agreed “facts” already established in the past.
Different publication forms focus on these sources differently. For example, a review summarizes past research but presents no new data nor is it expected to create a new theory. Theory papers also have no data but are expected to propose theory changes. A proposal is expected to review past research and by analysis derive a research question and method but no data is expected.
Stating publication type helps manage reader expectations. Identify the research type early, so readers know what to expect, e. g. don’t introduce a research proposal that has no data with words like “This paper investigates differences between …” as readers may be disappointed to find no data is given. If you present a new theory, then say so in the introduction so readers are prepared for that. The unexpected is normal in comedy and in the arts people like variety but in academia no-one likes to be surprised.
