We have to talk about chatgpt

[Cola]

Alright, I'm sure this wil anger a lot of people, but we have to talk about chatGPT, as it is getting to a point where it may be useful.

I am currently at a pathology conference, and the ability of chatGPT(v4) to recognise highly specialized medical images is getting very good.

Everyone is very hesitant to actually use it of course, but very often it is on par with what a skilled pathologist would say about some histology slides.

 

So obviously I was curious how it would do on some nihonto related challenges.

I went ahead and copied a partial description of Aoi into the chatGPT prompt excluding the era, and then asked it for the era.

 

This was the answer:

 

Based on the detailed description of the sword, several clues suggest the time period it could belong to:

1. Blade Length and Curvature (Sori): A blade length of 73.8 cm with a curvature of 1.8 cm is typical of katana swords from the Edo period (1603–1868). During this period, katana lengths averaged around 70–80 cm, and the curvature was relatively shallow compared to earlier periods, like the Kamakura or Muromachi.

2. Number of Mekugi Holes: Three mekugi holes are relatively uncommon, but it's possible this blade has been remounted multiple times, which might suggest it was used over a long period or refitted during the Edo period, when swords were often reused and maintained.

3. Width at Base and Tip (Motohaba and Sakihaba): The blade's width at the base and tip seems consistent with those of later periods when swords were made more for practical use (e.g., Edo period and onward), rather than for the battlefield, where earlier swords were heavier and broader.

4. Jigane (Hada): The presence of clear itame hada (wood-grain pattern) can be found across many periods, but its clarity suggests a well-forged blade, which was a hallmark of higher craftsmanship during the late Muromachi (1336–1573) or early Edo periods.

5. Hamon: The described Gunome-midare hamon with deep nioikuchi and Ko-niedeki features are also found in swords from the Edo period. The midarekomi boshi style (where the hamon pattern continues into the tip) became more common in the Edo period.

Given these features, this sword likely originates from the early to mid-Edo period (17th century). During this time, swords were crafted more for personal defense and status, with less emphasis on heavy, large battlefield weapons and more on elegant, practical designs.

 

Very curious to hear from the experts whether this is all a complete hallucination, or whether it actually makes sense. Aoi art states this example is from the muromachi period btw.

 

Secondly, it can do image analysis as well now; I gave it only the picture of this sword (https://www.aoijapan.com/katana：bizen-koku-osafune-sukesada-sakunbthk-hozon-token/)

 

And it did a very  detailed description; but wrong on several points:

 

From the image, I can make some observations about the sword's structure and key features:

1. Blade Shape and Curvature (Sori): The blade appears to have a gentle curvature, typical of many Japanese swords from the Edo period. This curvature often reflects a design more oriented for personal defense rather than the more pronounced curves of earlier tachi blades used in mounted combat.

2. Hamon Pattern: The hamon, or temper line, appears to be midare (irregular wave-like pattern), potentially gunome-midare, as it shows undulating peaks. This style of hamon became quite popular in the Edo period and is a hallmark of high-quality swordsmithing, where differential heat treatment creates these distinct patterns along the cutting edge.

3. Nakago (Tang): The tang has two mekugi-ana (peg holes), which can indicate the age and re-mounting history of the sword. Multiple holes often mean the sword has been re-mounted in different handles or styles over time. The presence of a signature (mei) on the tang is a valuable clue, as it typically contains the name of the swordsmith and possibly the location or date of forging.

4. Kissaki (Tip): The tip of the sword looks well-defined with a noticeable boshi (the hamon continues onto the tip). This feature, depending on its shape and continuation of the hamon, is crucial for identifying the sword school or period.

5. Sugata (Overall Shape): The blade appears to have a standard shape typical of katana rather than tachi, with a balanced curve and medium-length tip. This style is also indicative of Edo period katana, used for practical wear rather than the longer and more curved blades used in earlier warfare.

 

All in all, I think it's quite interesting that it has apparently been trained on nihonto specific text, and is to some extent able to recognise details, although right now not very accurately it seems.  

Perhaps this can be useful for some of us; possibly also for translation tasks. 

[PNSSHOGUN]

It gives a sword dated 1572 to the Edo period, that says it all.....!

[GeorgeLuucas]

I've used chatGBT a few times in this hobby, in an attempt to clarify some vocabulary that I struggled to differentiate. (Example: difference between hakikake and kaen boshi)

I found it a little helpful... One thing I've noticed and worry about, is that it pulls from ALL internet information. Both reputable and less reputable. I've used it with mixed success, but it's given me some pretty misleading and incorrect information too. 

All in all, I've stopped using it entirely when studying Nihonto. I found it confused me more than it helped, and the time it took me to fact-check it's information; was the same amount of time it would take me to learn the info from the source anyway. If I stick with Markus Sesko publications, for example, I get a much better and more trustworthy understanding of what I am looking for.

A crazy new world! I wonder what AI will be capable of in just 10 years. 
Cheers,
-Sam

PS. Using it to look at photos is an interesting exercise. I wonder if it takes information from for-sale listings; we all know how accurate those can be 

Edited September 10 by GeorgeLuucas
PS.

[Lee Bray]

This came up in the cast tsuba thread by 'Dan Tsuba' a while ago. I forget the exact question but the AI fabricated several books and articles for its answer.

So no, I don't think it's close to reliable yet. If it learns from the internet, will it ever be?

[Shugyosha]

7 minutes ago, Lee Bray said:

 If it learns from the internet, will it ever be?

 

I have my doubts, it will boil down to the classic garbage in, garbage out scenario. The problem that may come around is that the water gets so muddied that it's no longer possible to rely on internet-based research as no-one knows what sources are trustworthy or that it becomes accepted as reliable because, well it's the internet isn't it. We are cursed with living in interesting times.

[Tcat]

IMHO ChatGPT is terrible at providing accurate information / facts and not good at true analysis of abstract information. Absolutely everything it tells you needs to be checked. Its strengths lie in quickly generating high volumes basic format content which can then be checked and refined by a human. Trust the 'facts' it spits out at your peril.

[Cola]

44 minutes ago, Lee Bray said:

This came up in the cast tsuba thread by 'Dan Tsuba' a while ago. I forget the exact question but the AI fabricated several books and articles for its answer.

So no, I don't think it's close to reliable yet. If it learns from the internet, will it ever be?

 

Which version of chatGPT did they use? For medical challenges as well, GPT3 was hallucinating up all kinds of fake references, but GPT4 has gotten a lot better in that regard. 

But we also have similar worries about whether the model recognises outdated or fraudulent publications as false. 

 

One option that seems to help a bit (I cannot check if this works for nihonto since I don't have the required books) is to input the books you trust in the model, and to let it answer questions based on those inputs. 

[Lee Bray]

1 hour ago, Cola said:

 

Which version of chatGPT did they use?

I don't know, it wasn't mentioned, but the post was dated 24th May, 2023, if that helps.
Page 10 on this thread.

 

[Cola]

2 hours ago, Lee Bray said:

I don't know, it wasn't mentioned, but the post was dated 24th May, 2023, if that helps.
Page 10 on this thread.

 

That's got to be GPT3 then GPT4 is much more recent; and since he put the full convo in there I can just recreate it with GPT4; it's not very concise though.

 

Did Japan produce cast-iron ingots for commercial trade during the Edo period?

 

Japan did not widely produce cast-iron ingots for commercial trade during the Edo period (1603–1868). The Edo period, under the Tokugawa shogunate, was marked by a policy of sakoku (national isolation), which limited foreign trade and interaction. Japan did have a robust ironworking tradition during this time, but the production and trade of cast iron were primarily for domestic use, not for large-scale export.

Here’s a more detailed look at the situation during the Edo period:

Domestic Iron Production:

• Iron Production Techniques: The Japanese iron industry was based on traditional methods like tatara, a smelting process that involved producing steel (including tamahagane) from iron sand (satetsu). This method was labor-intensive and produced relatively small quantities of high-quality steel, primarily used for tools, weapons (e.g., nihonto), and agricultural implements. Cast iron, in contrast, is more brittle and less suitable for items like swords.

• Domestic Use of Cast Iron: Although Japan produced cast iron in small amounts, it was used mostly for specific items like pots, pans, kettles (e.g., tetsubin), and some larger construction or engineering components. Cast iron wasn’t considered a premium material, so its application was limited compared to wrought iron and steel, which were more highly valued for their durability and workability.

Commercial Trade and Sakoku Policy:

• Limited Trade: Under the sakoku policy, Japan only traded with a few nations, primarily China, Korea, and the Netherlands (through the Dutch East India Company in Nagasaki). The scope of this trade was tightly controlled, and there is no significant evidence of large-scale exports of cast-iron ingots during this period. The iron trade primarily focused on domestic demand rather than export markets.

• Iron Imports: Japan actually imported iron ore and other raw materials during the Edo period, primarily from China and Korea, to supplement its own iron resources. This further indicates that the country was not in a position to produce surplus cast-iron ingots for export.

Why No Large-Scale Cast-Iron Export?

1. Economic Priorities: The Tokugawa shogunate prioritized self-sufficiency and maintaining domestic production to support agricultural and artisan communities. Iron production was focused on fulfilling internal needs, especially for farming tools and construction materials, rather than generating export surpluses.

2. Technical Limitations: Japan's tatara smelting technique, while excellent for producing steel, was not geared toward mass-producing cast iron in ingot form for trade. Other nations, particularly in Europe and China, had developed larger-scale iron foundries that could mass-produce cast iron more efficiently.

In summary, Japan’s production of cast iron during the Edo period was limited to meeting domestic needs, and it was not a major player in the international cast-iron trade during this time.

 

Provide references for these statements:

 

During the Edo period (1603-1868), Japan was known for its production and trade of various iron products, including cast iron. While Japan is not particularly rich in natural iron resources, it developed a significant iron industry, especially in regions like Mimasaka, Bizen, and Bitchū, where iron was refined and made into products like tools, swords, and other artifacts. Some sources mention that iron was cast into ingots and traded domestically, though Japan's primary focus was on the refinement of iron sands into steel for the production of swords and armory(

Gunbai: Ancient Japanese Warfare)(Gunbai: Ancient Japanese Warfare).

Commercial export of iron was limited, primarily because Japan's isolationist policy (Sakoku) restricted foreign trade to a few select ports. However, Japanese merchants did engage in limited foreign trade, particularly with the Dutch and Chinese, who exchanged iron and copper. Japanese cast iron foundries, particularly in regions like Oshu, specialized in producing high-quality iron products for both domestic use and trade(Boonie Hicks)(Gunbai: Ancient Japanese Warfare).

 

Provide references about the production of cast-iron ingots in Japan during the Edo period

 

The production of cast-iron ingots in Japan during the Edo period (1603–1868) was not a major commercial focus. Japan primarily produced iron for domestic consumption, with iron sand (satetsu) refined into higher-quality steel for tools, weapons, and other local uses. Regions like Mimasaka, Bizen, and Bitchū were known for iron production, though large-scale ingot production for international trade was uncommon due to the country’s isolationist policy (sakoku)(

Gunbai: Ancient Japanese Warfare)(Gunbai: Ancient Japanese Warfare).

Iron was smelted using traditional techniques like tatara smelting, which primarily aimed at producing high-quality steel (like tamahagane) rather than mass production of cast iron. The limited cast-iron products, such as kettles and other domestic wares, were largely for local use. Even though some cast iron was produced, it wasn't part of large-scale commercial trade, especially since Japan imported iron ore from China and Korea to supplement its own supplies(Boonie Hicks).

For further insight into traditional Japanese iron production, including casting, the regional foundries like those in Oshu are still known for their historical contributions to the production of high-quality cast iron(Boonie Hicks)(Gunbai: Ancient Japanese Warfare).

 

[GeorgeLuucas]

https://www.militaria.co.za/nmb/topic/47942-ai-and-nihonto-chatgbt/#comment-497930

Here's a thread I started in the Izakaya in February about ChatGBT, with the prompts and answers that I fed it. 
Cheers,
-Sam

[Ghoul]

I'm not a nihonto expert, but I work with AI language and image models regularly as part of my job and hobbies, and there's a lot of misinformation about what AI can and can't do...

 

Language models do not know things. They simply create text that reads like the most likely answer based on the previous context it has access too.

 

This means it is very good at creating things that sound right, but not things that are accurate, or would disagree with its previous context. 

 

Yes, you could create a more accurate model using only reputable sources (e.g. adding only Sesko's and others work, giving it none of the mess that is online sources (this would be stealing without permission though)). But it has no ability to reason or solve conflicts of information - it would simply try to create what best fitted the theme of the works previously provided, leading to decent accuracy but regular terrible mistakes that even a beginner would catch.

 

This isn't to say that AI can't be used in the field of Nihonto though - I actually have high hopes with enough images and sources image models would be very useful at matching signatures against indexed swordsmiths. Similarly, the initial terribly inaccurate assessment can be used to speed up writing a human analysis or text, as it at least provides a starting point, however wrong.

 

In general AI like most tech tools thrive on very specific datasets and tasks - general use (like trying to analyze all things about a sword), not so much.

[John C]

Just my two cents - I think AI can eventually be useful in Nihonto identification, however similar to the way it is trained to spot cancer, known valid examples will have to be offered so it can "learn" what to look for. In breast cancer, for example, thousands of scans that were identified by the doctor as normal but where cancer was later identified in the individual, were used to train the AI. Currently, it can identify cancer not identifiable to the radiology technician (I forget what the accuracy rate is).

I think an area where it could be most useful would be in mei authentication.  If it can be shown enough scans of legitimate mei for a given smith, it could learn the subtleties of his chisel marks or whatever. Access to this program could be used by shinsa or by sword buyers just wanting to confirm a signature. 

The problem will always be garbage in; garbage out. There would need to be certainty that the mei used to train the AI are legitimate.

 

John C.

 

[Bazza]

Just a 'simples' question - does any know or care to guess at the carbon footprint of chatGBT???  Or the internet???

 

BaZZa.

[Lee Bray]

Rather large, Bazza. Knew it was big but just googled and they reckon it uses the same amount of power as Sweden currently and will rise to Germany's supply by 2026, roughly 500 terawatt hours rising to 1000 terawatt hours. That's AI alone, no internet.

 

 

[Shugyosha]

If you add in Bitcoin and the other crypto-currencies, it starts to get a bit scary:

 

https://www.imf.org/...-tax-policy-can-help

[raynor]

Speaking as someone with an education in the IT field - Large language models like ChatGPT are getting more accurate every day, but I firmly place them within a rule as old as the internet: Do not take serious medical or academic advice from an online generated source, chatGPT and its like should be treated as a fancy version of wikipedia, nothing more.

[Cola]

This one was too funny to keep from you: