Integrate optimized image based GEMM kernels for Intel

[gongzg]

This PR implemented an image based blocking matrix multiplication kernel to do the GEMM on Intel platforms. And also choose a build time platform selection for now, as the OCL driver doesn't provide detail information.

[gongzg]

@ptillet please review this PR. thanks. BTW, I will be on vacation in the coming two weeks. @listenlink and @wujunkai166 will have some time to work on this in next week.

[ptillet]

Hello,

Thanks for the new pull request, it looks much better and easier to integrate. A few comments:

• Would it be possible to determine SKL/BDL through a CPUID check at runtime? I want to keep the installation/packaging as straightforward as possible. If it's determined at compile-time, it will not be possible to distribute a performance-portable version of Isaac.
• I'm still kind of skeptical about the .json file for SKL, I've had relatively good experience with the prediction mechanism on Intel/NVidia/AMD platforms, and it strikes me as odd that it wouldn't work for image-based kernels. The auto-tuner should work out-of-the-box even for external profiles and shouldn't require any change. I'll test it on my platform to see what's preventing it from being usable with image-based kernels.

Otherwise, it looks good :)

Happy Chinese New Year @gongzg

[gongzg]

@ptillet The more reliable way to determine GPU information is to check the iGPU's PCI ID. Actually, the ideal way is to get the necessary information by query the OCL device information by using OCL APIs only. Beignet could report what we needed, but the closed source driver couldn't. I am discussing with the closed source driver team to get the required detail information rather than the current generic information. If finally, both drivers could give the necessary device information, we can query it at runtime gracefully. But if finally, we still can't get it at OCL API level, we may have to check the PCI ID. For now, I think the build time option is an acceptable method.

As to the SKL .json file, @wujunkai166 found once enable other profiles, it will bring an obvious performance regressions with the alexnet's batch size 128 on SKL platforms.

[ptillet]

Yes, the OpenCL driver would be the best way. That said, I am generally very opposed to build variables; I think it puts on the user a burden that should be on the developer. Not to mention that most people will set USE_BROADWELL or USE_SKYLAKE according to the CPU model and not the GPU model. Checking the CPUID is not very safe I agree (because a newer generation CPU might use an older GPU), but it should reproduce the behavior or 95% users.

I get significant performance regression on my Kaby Lake laptop when always using the image profile (not to mention the large memory consumption). Details here: https://paste.ubuntu.com/23839685/. ISAAC tends to perform better on NN layouts, as well as NT except for small K, but the Intel kernels perform 2x better on TN layouts, so that's very good. I can add an environment variable to bypass the prediction mechanism (benchmark all and cache in RAM) in the case of Deep-Learning applications, so that the new .json file will introduce no performance regression on Caffe. An additional CMake variable would also be very appropriate in this case, to make sure that clCaffe always bypasses the random forests.

I will investigate that CPUID thing over the week-end, as well as that empirical prediction mechanism.

[gongzg]

@ptillet I'm fine if you want to use CPUID at this stage. :), please go ahead. Thanks. BTW I'm afriad that I can't response from now on until I back to work.

[ptillet]

Sure, no problem! Have good holidays and happy new year :)

[ptillet]

(b0d6ec4 should take care of detecting the iGPU architecture at runtime.)

[listenlink]

@ptillet , thank you for your quick implementation of CPUID. there are some conflicts with the PR now, I'll fixed it soon. But the PR status can't be updated until @gongzg's back :)

[gongzg]

@ptillet I just rebased the PR with latest master. Please review again. Thanks.

[ptillet]

Very good. At this point I'm ok with merging the cpp files.

I still think we need to sort out the issue with auto-tuning. There are shapes for which ISAAC kernels perform better than the image kernels on my Kaby Lake laptop, although sometimes the latter are up to 2x faster. I think that the pull-request should only include small changes in tune/tools.py (similar to what is done for cublas_gemm), and then I could possibly had an environment variable to disable the predictive mechanism.

[gongzg]

I'm fine if only cpp files are merged at this time. After the FP16 development, we will look into the tuning issue as well. Thanks.

[gongzg]

@ptillet We will regenerate jason files for BDW and SKL by using default tuning mechanism and remove those hacking in the script as well. Please hold and wait for us to submit another new version of code.

[ptillet]

Okay, that looks good to me :)
Once the this pull request is merged, I'll do some extensive testing to make sure there is no performance regression anywhere, and will add a new stable release.