self-deep learning

play_image Train GPT on images ¶ Effectively re-implements OpenAI's Image GPT model, getting GPT to model images instead of text, but using a near identical model. It's truly quite remarkable that a single model can agnostically do a great job modeling whatever data you give it: text, images, or whatever else. At the end of the day it is just a sequence of integers. Notice that unlike models like PixelCNN++ etc, this model knows nothing at all about the spatial layout of the pixels and has to learn the appropriate positional embeddings that reflect the spatial topology of the data. github: https://github.com/karpathy/minGPT minGPT 코드 이해를 돕기 위한 노트북. attention 설명 생략 전체 코드 및 참고 자료는 Andrej karpathy github 참고 바람 play_image 목적 image 데이터를 학습하여 새로운 image 생성 -> how?: image 데이터도 결국 숫자로 이루어진 데이터에 불과. 이를 sequential data로 변환하여 transformer 모델에 적용. (즉, 다음에 나올 pixel value 예측) image 변환(sequential data) -> Model -> image 예측(se...