. | . | . | . | ||||||||||||||||||||
. |
Description of studyThe study attempted to measure attitudes about Talisman2 and about academic scholarship in general to determine whether these constructs might be usefully measured and whether a relationship would exist between the two constructs. I supposed that positive attitudes toward scholarship would correlate to positive attitudes toward Talisman2.Two pilot scales of six items each were sent as a 12-item survey to about 60 e-mail addresses listed as subscribers to the Talisman2 newsgroup. An additional four surveys were sent to persons historically involved on Talisman who were not currently subscribing members. Thirty five persons responded to the survey, with four persons declining to participate and 31 persons answering the survey. Seven addresses seemed not to work, and about 20 persons never responded in any way to my surveys. All identifying information was removed from responses, so I do not know which 31 addresses answered the survey and which four did not, nor do I know who wrote comments. Responses came from frequent participants and from persons who had never posted (to my knowledge). Frequent participants and lurkers were also represented among the persons who never responded in any way the survey. The sample was too small to accurately determine reliability, although
measures of internal consistency generated from this small sample indicate
the scales are promising (Cronbach Alphas above .80). This page discusses
the items of the scales and responses to them.
The survey items and responses1) By providing responses and debate with difficult issues or questions, Talisman2 helps prepare Bahá'ís to be better teachers of the Faith.Agreement with this item shows a willingness to make a favorable generalization about Talisman2.Comments from surveys
Response to CommentsAny Bahá'í group or event will have a range of influences on the teaching spirit within Bahá'í participants. This question asks respondents to agree with a generalization that could clearly be true or untrue in any specific case. Would discussing issues that cause discomfort help some of us improve our ability to teach? Do people enjoy impersonal forums such as e-mail newsgroups and mailing lists because personalities in local communities have not created a safe environment to discuss controversial or discomforting issues. Would this mute our teaching voice? Does tolerating non-comformist views help us teach? These are questions that could be studied by combining an empirical survey approach with study of Bahá'í scriptures and history. Further discussionMean: 4.3; StdDev: 4.7; Skew: -1.1; Kurtosis: 1.7 A clear majority of this sample agrees that Talisman helps improve teaching ability. However, responses tend to be half way between complete agreement and neutral, indicating qualified agreement. The distribution of responses suggests this item or one similar to it could be usefully adapted in any measure of how Bahá'ís perceive a Bahá'í group or event is preparing Bahá'ís for teaching the Faith. However, the question is double-barreled. Had I simply asked for respones to
the statement Talisman2 helps prepare Bahá'ís to be better teachers of the
Faith the question would have been more general. I decided to add the clause about
debate with difficult issues or questions because I wanted to know if
people specifically thought this was a source of benefit from Talisman.
Obviously Talisman2 is much like any gathering of Bahá'ís in many ways, and
would presumably influence teaching as much as typical Bahá'í groups
inasmuch as it was similar, but Talisman2 is most notorious (I suppose) for
the difficult issues addressed freely there.
2) Some individuals on Talisman2 have a deliberate agenda to undermine respect for Bahá'í institutions.Agreement with this item shows a negative attitude toward Talisman, so disagreement is positive toward Talisman.Comments from surveys
John Cornell wrote: Response to CommentsA key word in the question was *deliberate* since people could have agendas, (conscious or unconscious) that indirectly undermined respect. People could hypothetically have a deliberate agenda to defend the Faith, but a consequence of their defense could be undermining the Faith, as was the case when Bahá'ís murdered Covenant Breakers or when Babis tried to assassinate the Shah. I guess my assumption was that most Bahá'ís would feel unfavorable toward any group or event in which some persons participating were perceived as trying to undermine respect for authority. I agree with you that this assumption is questionable, because some of us may have favorable feelings toward groups or events even if we think some people participating are "up to no good." One advantage of creating a scale using several items is that the validity of each item may have limits, but we can assume that what is valid about each item is measured by the scale as a whole. At least we can do this when all the items are put togeth er and factor analysis shows that people tend to answer the items as a whole with reliable patterns of responses. If people conduct surveys with only a few questions and report findings on individual items instead of on scales or factor scores, then the p roblems of validity for each item make us more suspect of the findings. You make an important point how negative feelings about persons or agendas may not be included in our opinions about the community as a whole. If we wanted to make a scale about Bahá'í Love in communities and we might include questions like: 1) some people in my community show by their actions that their hearts are overflowing with love for me.
or The first question might be scored positively on the scale, but someone could feel there were a few good Bahá'ís in a community, but that the community as a whole was cold and unloving. Likewise one could think some of one's co-religionists were a pain in the neck, but think the community as a whole was very loving. I suppose one could try to force people to think of the whole, rather than the people within the whole: 1) One could characterize my community as a family in which people's hearts are overflowing with love for me. 2) One could characterize my community as a group in which people are always trying to stir up trouble. 3) It is fair to characterize Talisman as a mailing list existing to support an agenda of undermining respect for Bahá'í institutions. Would phrasing questions like this get around your problem, John? They might be more valid, but I wonder if the responses might be skewed with too many people giving extreme "disagree" answers. I think that survey items such as #2 are not perfectly valid, but they are probably valid enough to be used in scales (rather than as stand-alone items). Further discussionMean: 1.3; StdDev: 7.9; Skew: -0.3; Kurtosis: -1.6 This question elicited the most polarized responses. Eleven in the sample disagreed with scores -5 or lower, and 14 agreed with +5 or higher. Only six responded with more neutral answers between 0 and 4. The question shows a high kurtosis. (More scores than you would expect at the ends of the curve, and not as many as you would expect at the middle of the curve). The fairly equal distribution of answers on each end, with modes at the anchors of perfect agreement or disagreement means this item helps distinguish between favorable and negative attitudes toward Talisman. It is possible that people could perceive a deliberate plan to undermine respect for authority and consider that this was a good thing. My conception of favorable attitudes toward Talisman2 assumes the sample shared a frame of reference in which deliberate undermining of respect for Bahá'í institutions would be a bad thing. In research within the Bahá'í community we have to be careful of our
assumptions. I think we can assume that Bahá'ís perceive undermining respect
for Bahá'í institutions to be a negative thing. There are other assumptions
we could make in wording our questions that would not be as safe. 3) Persons literate in Persian, Arabic, and English have a special advantage in understanding the Bahá'í Faith.Agreement with this item shows a positive attitude toward academic scholarship.Comments from surveys
Response to CommentsI set anchors for responses at complete and perfect agreement or disagreement. Evidently some people had more than complete disagreement with some statements. Further discussionMean: 4.8; StdDev: 4.2; Skew: -1.6; Kurtosis: 4.2 The sample tends to agree that knowing the languages of revelation and authoritative translations helps. However, only one person gave complete agreement, so again, people recognize that there are limits to how much language helps. If a question like this were reduced to a yes/no response we would lose the information about the range of agreement. The high kurtosis suggests there is not much variance on this question, as responses are closely grouped together between 5 and 9. The item remains useful because the scale is a broad range of responses. If such questions are constrained to yes/no we would get virtually useless distributions of 2 no, 3 abstain, and 26 yes responses. The implication is we should not constrain scales to three, or even five categories of response. Scholarship depends on people being able to read sources. The academic scholars assume that understanding is improved when one can read souces in the original language. Scepticism of scholarship might manifest itself in the belief that knowledge of languages used in original texts is of insigificant importance, that true understanding requires only a pure heart. I want to emphasize that negative responses to this item do not seem anti-intellectual to me. Consider that among the worst enemies of the Bahá'ís are the clerics and politicians literate in Persian, Arabic, and English who have condemned us to death and jail in Iran and other countries over the past 150 years. Certainly we agree these persecutors can not possibly understand the Bahá'í Faith. Some of the Hands of the Cause did not understand Persian and Arabic. Questions about the usefullness of knowing the languages of the Revelation should not be confused with questions about intellectual or anti-intellectual trends in the Bahá'í Community. This scale of scholarship should measure attitudes toward academic scholarship and scholars, and not simply intellectual orientations. One more thing about this item, factor analysis showed this had the lowest
loading on the scholarship scale's principle component. This may just be
an
artifact of the small sample, but it could also mean that this item has the
least validity in measuring whatever the other five items measured (attitudes
toward scholarship).
4) I am deeply worried by much of what is posted on Talisman2.Agreement with this item represents a negative attitude toward Talisman2.Comments from surveys
Response to CommentsAs answers were still coming in someone pointed out in a Talisman post that this question did not address what sort of posts and opinions on Talisman would worry us. The scale was supposed to measure favorable attitudes toward Talisman2, and was not specifically supposed to measure the causes or ideologies behind the attitudes, whether favorable or unfavorable. People could be worried about tone, content, words used, or questions asked. If questions were phrased to include ideological frame with attitude toward Talisman it would be difficult to use results to examine relationships between ideology and attitudes. With a longer survey one could have several items combining ideology or cause with the emotive response to Talisman2, and thus have two or more scales (e.g., Scale to measure discomfort with content on Talisman and a scale to measure discomfort with tone of posts on Talisman). But the six item scale I used was only supposed to measure the general feelings toward the mailing list. For this reason question 4 is actually superior to questions 2 and 6, which could be measuring both Talisman attitudes and a personality dimension of trust versus suspicion, or ideologies about literalist understandings of doctrine Further discussionMean: -1.0; StdDev: 6.7; Skew: -0.0; Kurtosis: -1.4 Three groups emerge. Seven people are definitely worried about Talisman, 12 are not worried, and 11 have neutral to mild worries about it. Perhaps we should be surprised that only seven in the sample expressed strong agreement with this item. There could clearly be a sampling bias, as persons who are most worried about Talisman are going to unsubscribe or never subscribe in the first place. Persons most worried about Talisman may also have been more likely to refuse to participate in this survey. I would not generalize findings of how this sample answered this item to the wider Bahá'í community. The first time I mentioned Bahá'í newsgroups and mailing lists to another Bahá'í my friend's first response was that some of the things being said on the Internet scared them. This friend happened to be a staff at a national center, and I've often wondered what scared them. Another time I mentioned Bahá'í newsgroups to a Bahá'í friend (a member of an important Bahá'í committee with international responsibilities); again the first response was that many unholy things were being said in Bahá'í Internet space. I'll admit that I am worried, although only slightly, by both content and tone of some posts in Soc.Religion.Bahai and Talisman2. I am still not sure what implications to take from the recognition of unholy and frightening communication in Bahá'í Inter
net communties. I've made public and private posts a few times over the years to beg or encourage people to use sweeter words, but one has to wonder if the impersonal nature of webspace and the relative social immaturity of humanity (including Bahá'ís of
all ideological persuasions) will continue to cause consternation among the tender-hearted on the Internet.
5) Recognizing their limits, I still believe modern techniques of scholarship should be applied in general to studies of the Bahá'í Faith.Agreement with this item manifests a positive feeling about scholarship.Comments from surveys
Response to CommentsI thought some people would think techniques of scholarship were not appropriate for studying the history or community health of the Bahá'í Faith. It seemed to me that certain elements of the scientific method as practiced today, especially Occam's Razor (using the most parsimonious explanation for events), would be rejected by Bahá'ís in some circumstances, and not out of some anti-intellectual reaction against science, but out of a need to make room for God and spiritual reality in the face of scanty empirical evidence. Further, there seem to be so few empirical studies of the Bahá'í Community as it exists now I had assumed many Bahá'ís did not want to subject their projects or communities to the eye of social science research. Further discussionMean: 7.3; StdDev: 3.3; Skew: -1.7; Kurtosis: 2.7 All but three of the sample agreed with item five, and only one expressed disagreement. Ten persons had perfect agreement with the statement. Clearly in this sample almost everyone agrees that modern methods of scholarship can (and should) be applied to studies of the Bahá'í Faith. The responses to this item have somewhat high skew and kurtosis. Perhaps there is not enough variance on this item. Also, the item may be too obviously correct. The item could be rephrased to be a bit more extreme to help reduce the skew. For example, one could suggest more controversial modern methods of analysis such as deconstruction, or one could replace the word "should" with the word "must". Daniel Rigney and Thomas J. Hoffman, in their 1993 article "Is American
Catholicism Anti-Intellectual?" used an item from the 1988 General Social
Survey to measure attitudes toward freedom of scientific inquiry. The item
they used was: "Scientists always seem to be prying into things that they
really ought to stay out of." Perhaps this would have been a better item to
measure attitudes toward academic scholarship. (See the Journal for the
Scientific Study of Religion, Vol. 32, No. 3, pp. 211-222 for the Rigney and
Hoffman article).
6) Talisman2 is a tool for critics of the Bahá'í Faith.Willingness to agree with this generalization about Talisman indicates a negative attitude toward the mailing list.Comments from surveys
Response to CommentsThe first comment raises validity questions about this item. Some participants in Bahá'í newsgroups and mailing lists are not Bahá'ís can not be expected to share Bahá'í concerns about criticism. Further, authoritative Bahá'í texts sometimes emphasize the benefits to the Bahá'í Community when it faces criticisms. Some Bahá'ís may therefore consider that newsgroups or mailing lists used by critics of the Bahá'í World Order (such as it is) are good places to exchange views. Such Bahá'ís may have a favorable attitude toward arenas for discussion between Bahá'ís and our critics. Further discussionMean: 1.1; StdDev: 7.3; Skew: -0.3; Kurtosis: -1.2 Seven responded with strong disagreement (-8 to -10) and nine responded with strong agreement (8 to 10). There is no consensus on this item. The item is flawed in that people could agree with the item as factually correct yet disagree with the item as a good generalization about the newsgroup. The item could have been more specific if worded: "It is a fair characterization of Talisman2 to say it is basically a tool for critics of the Bahá'í Faith." Or, to swing the item in another direction the item could have been worded: "Leaving aside the question of whether it is a fair characterization of the group as a whole, I believe that it is a fact that Talisman is used as a tool for critics of the Bahá'í Faith." Since the mean for the item as it is worded is close to the middle between the anchor points I suppose these more precise wordings would have moved the scale in a direction of higher skew (less symmetry in the responses) In constructing scales to measure Bahá'í attitudes toward their community or
their religion one must either give specific statements or broader
generalizations. The broad generalizations are shorter, easier to read, and
cause less confusion, but people who agree with the generalizations as
specific statements of fact may decide to express disagreement with the
items because they do not think the items are good generalizations. Thus one
must be careful in interpreting results. For example, persons who disagreed
with item 6 may still have believed critics were using Talisman2. That is, they
could have believed critics *were* in fact using Talisman as a tool, but may have
(correctly) understood the scale was looking for generalized feelings about the mailing
list, and so they may have answered with a negative response (if they felt the
the general character of Talisman was not set by the few persons or posts perceived
as "tools of critics".) Further, persons who answered positively to item six may have felt a favorable attitude toward Talisman2 if they enjoyed participating in a newsgroup they perceived as being a tool of critics. These sorts of answers bring into ques
tion the validity of items like this.
7) Modern scholarship tends to be too materialist to be of much use in studying the Bahá'í Faith.Agreement with this item shows a willingness to make a skeptical generalization about academic scholarship.Comments from surveys
Response to CommentsThe second comment explained the only missing answer in the whole survey. I replaced the missing value with the mean of the other 30 answers, which was -3. The second comment also illustrates the frustration I had with many fields of study. I went into social work research because I feel at least in this field we are trying to help people develop better systems of help, and so we are looking for what works just as much as looking for ways to criticize what doesn't. Much as in the Bahá'í Faith, we social workers have goals and a code of ethics, and direct our research towards helping people develop towards the ideals. But of course, it's fun to read a good critical works. Whether it's Tacitus describing the Roman emperors, Shoghi Effendi explaining the situation in 19th Century Iran (or American society), or Harry Specht criticizing the social work profession, or James Loewen taking on American high school history textbooks.. Oh I could go on with all the great critical works I've enjoyed lately, but really, when one is examining the fruit of a human civilization alienated from the human heart, what can one expect but a lot of criticism? Further discussionMean: -3.6; StdDev: 6.1; Skew: 0.5; Kurtosis: -0.9 This statement is very close to item five, but the statement is expressed
negatively rather than positively, and it is a bit more extreme. As with item
five there was a clear mode at the extreme pro-scholar anchor point. The
distribution of this question helps seperate scores more than did item five.
A question such as this would probably be better than 5 in any scale
measuring attitudes toward scholarship within the Bahá'í community.
8) Talisman2 offers participants a chance to deepen their understanding of the Bahá'í FaithAgreement with this item shows a willingness to make a favorable generalization about Talisman2.Comments from surveys
Response to CommentsPeople wanted to be able to qualify their agreement to this question. This is another example of why we should not constrain answers in surveys to simple yes/maybe/no responses. People have a wider range of agreement or disagreement. Further discussionMean: 5.5; StdDev: 5.9; Skew: -1.7; Kurtosis: 2.0 Only five persons disagreed with item 8, and this was the only item for which no one answered with a neutral 0 response. The statement seems too qualified in that it only says Talisman2 offers a
chance and does not say directly that Talisman does in fact help Bahá'ís
deepen in their understanding. Changing the wording to make the item less
qualified might move scores more toward the neutral or negative responses,
thus reducing skew.
9) A person with a strong academic training in theology and history would tend to have better informed opinions about the Bahá'í Faith.Agreement with this item shows a positive attitude toward scholarship.Comments from surveys
Response to CommentsThe first comment points out my assumption that people may value a better informed opinion when in fact what people tend to value is a wise opinion. Many of the Bahá'í Faith's most vocal critics are clerics or missionaries of other religions (who presumably have some training in theology) so clearly disagreement with item 9 should not be equated with an anti-intellectual orientation. Further discussionMean: 1.9; StdDev: 6.3; Skew: -0.3; Kurtosis: -1.0 From the comments I get the impression this dimension of respect for scholarship could be measured with several items. We have to remember there are differences between educated and well-informed opinions versus wise and intelligent opinions, since clearly persons with dull intellects have received advanced degrees. After all, there were doctors with MDs willing to support the death camps of the Nazi empire and experiment in them. And isn't one of the alleged war criminals of the former Yugoslavia who whipped up the frenzy for genocide a psychologist with an advanced degree? Educational attainment may correlate to intelligence and wisdom, but it is a poor measures for those; it also has limits as a measure of intellectual orientation. Academic achievment is probably a good measure of orientations toward intellect, but there are limits to its validity. Persons may become well educated for economic advancement. Intellectuals may prefer to not attend universities or pursue academic degrees. Persons with the money and a class background that allows them to attain higher education may tend to have their perceptions biased by a sense of allegiance to ideologies and systems that helped earn their degrees. The process of education may also have socialized them to narrow their thinking. For example, throughout American involvement in the Viet Nam civil war persons with college educations were always more hawkish, while more persons with merely high school education or grade school education supported American troop withdrawal. Since today nearly everyone agrees the Viet Nam War was a mistake (even people who waged it like Robert McNamara) it is clear that persons with less education had wiser perceptions, at least on that issue. Any measures of the intellectual climate within the Bahá'í Community should
go beyond simple counts of how far Bahá'ís tend to advance in formal
education to earn degrees. Such information must be supplimented with survey
data showing the attitudes toward learning and intellect among Bahá'ís.
10) To the extent that Talisman2 offers any picture of the Bahá'í Faith it offers a distorted and inaccurate one.Agreement with this item shows a negative attitude toward Talisman2Comments from surveys
Response to CommentsThe tone of posts may not be representative of the typical styles of Bahá'í communication. When people post things about the Faith or dynamics within the community their experiences and preceptions may tend to be inaccurate. On the other hand, the diversity of views, the methods of expression, and the perceptions shared on Talisman could be representative of what one would experience in a typical Bahá'í community. This is an item where I expected few responses to be at either extreme anchor point. Further discussionMean: 1.4; StdDev: 6.5; Skew: -0.5; Kurtosis: -1.1 As I expected, answers are all over the place on this item, without a clear cluster anywhere. The average was close to neutral. The question may have validity problems. People may have a favorable or
unfavorable attitude toward Talisman regardless of whether they believe it
is an accurate representation of the Faith. My assumption was that opinions
that a mailing list was accurate would be favorable and opinions that it gave a
distorted view would be negative.
11) Scholars tend to be blinded by their acquired knowledge or pride.Agreement with this item shows lower respect for academic scholarship.Comments from surveys
Response to CommentsIt is of course true that anyone can be blinded by acquired knowledge and pride. An interesting question is whether we think scholars are more likely to be blinded. Scholars could be less likely to be blinded by knowledge if one believed academic training and study tended to make people more open-minded and humble. On the other hand, scholars could be at greater risk for a kind of intellectual pride that would blind them to truth. Since this question is part of a scale that is supposed to measure attitudes toward academic scholarship in general, answers should reflect how people think about Bahá'í and non-Bahá'í scholars. Further discussionMean: 0.8; StdDev: 6.2; Skew: -0.4; Kurtosis: -1.0 Only eight persons disagreed with item 11, and all eight gave answers below
-5. The item could have been improved by making it a little more extreme so
more people would have disagreed with it. I would suggest a new wording of:
"Compared to others, scholars are more likely to be blinded by acquired
knowledge and pride." It would be interesting to see if there are differences
in the degree of concern Bahá'ís have about spiritual pride and acquired
knowledge blinding Bahá'ís in administratrive positions, teachers, scholars,
pioneers, etc. Concerns about acquired knowledge and pride could be rooted
in experiences with persons in those positions or in suscpicious and cynical
attitudes among believers, quite possibly both. I hope people would find
implications for their behavior if they found a significant number of
Bahá'ís perceived their co-religionists in academic professions or in
administrative positions as being blinded by their learning and pride.
12) The Bahá'í Faith would benefit by suspending the policy of publication review for at least those articles submitted for publication in peer-reviewed scholarly journals.Agreement with this item shows a confidence in the ability of scholars to use wisdom in publishing, and is scored as a positive attitude toward scholarship.Comments from surveys
Response to CommentsI am surprised to hear that Bahá'ís with ideas as radical as those expressed in the first comment would have the maturity to submit their work for Bahá'í publications review and the audacity to submit such unusual papers to juried academic journals. Further discussionMean: 3.7; StdDev: 7.1; Skew: -0.8; Kurtosis: -0.8 This scale had the mode with the highest frequency, since eleven persons completely agreed with the statement. This item was carefully worded so that agreement with it would not necessarily mean
people were agreeing to the idea that the policy of review should be
suspended. Bahá'ís tend to avoid making statements in which they tell
administrative bodies what they should do. Phrasing the statement to state
that there would be benefits to removing review in specific types of
situations probably allowed more agreement than we would find with a simple
statement that "review should be repealed." ScalesWith only 31 subjects it is difficult to interpret results, especially Cronbach Alphas and Principal Component Analysis, which require hundreds of subjects for appropriate use. However, as a pilot study, these two scales look promising. Both scales were made by summing scores from six items discussed above, with scores reversed to that in both cases positive scores represent positive attitudes. Scales had a potential range from -60 to +60. The scale measuring attitudes toward Talisman2 shows a mean of 7.0, which is very close to neutrality (standard deviation was 29.5). Evidently one can generalize to say this sample had mixed feelings about Talisman2, without a clear degree of positive or negative opinions about it. The Cronbach Alpha for raw scores was .84, which is high for a six item scale. Factor loadings in principal component analysis showed all items had strong to moderate loadings on a single principal component. The scale measuring attitudes toward scholarship had a mean of 20.5, and a standard deviation of 24.3, which suggests that this sample's attitudes toward academic scholarship, as measured by this scale, tends to be positive. The Cronbach Alpha
for raw variables was .81, which is high for only six items.
Factor loadings were: The scale on Scholarship: A bivariate analysis shows a fairly strong relationship between scores on the attitude to scholarship scale and the attitude toward Talisman scale. The scatterplot below shows how well the data fit the model.
|
. | |||||||||||||||||||||
. | . | ||||||||||||||||||||||